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DEVELOPING  PERFORMANCE  MEASURES  FOR  ARMY  AVIATION  COLLECTIVE 
TRAINING 

EXECUTIVE  SUMMARY 


Research  Requirement: 

Assessment  systems  are  an  essential  element  of  effective  training  solutions.  As  a  result, 
it  is  of  critical  importance  to  develop  performance  criteria  for  aviation  collective  tasks  in  order  to 
provide  feedback  to  aircrews  and  to  enable  leaders  to  monitor  the  progress  of  the  unit,  diagnose 
and  remedy  training  deficiencies.  This  research  was  intended  to  provide  prototype  measures  of 
Army  aviation  collective  task  performance  in  attack-reconnaissance  missions  as  currently 
conducted  in  theater. 

Procedure: 

The  aviation  training  exercise  (ATX)  is  conducted  in  a  networked  virtual  environment  at 
the  U.  S.  Army  Aviation  Warfighting  Simulation  Center  at  Fort  Rucker,  AL.  Limiting  our 
efforts  to  the  reconnaissance-attack  mission,  we  examined  the  utility  of  observer-based  and 
automated  simulator  (i.e.,  system-based)  data  as  measures  of  collective  performance  at  all 
possible  points  during  the  simulation.  First  a  set  of  critical  tasks  was  defined.  Next,  indicators  of 
high,  average  and  low  performance  on  these  tasks  and  underlying  skills  were  developed.  Finally, 
measures  were  developed  to  quantify  task  performance  and  to  provide  systematic  feedback. 
These  steps  were  accomplished  in  an  iterative  series  of  three  workshops  in  which  subject  matter 
experts  collaboratively  worked  with  behavioral  scientists.  The  measures  were  based  on  tasks 
commonly  performed  in  Attack  Weapons  Team  or  Scout  Weapons  Team  missions. 

Findings: 

Performance  indicators  for  five  mission  phases  and  further  broken  down  into  12  mission 
events.  A  set  of  44  performance  indicators  and  101  supporting  performance  indicators 
(observable  behaviors)  were  identified  that  captured  collective  performance  during  critical 
events.  Based  on  these  observable  behaviors,  a  total  of  1 15  observer-based  measures  that  could 
discriminate  high-performing  from  low-performing  teams  and  that  provided  behaviorally-based 
feedback  were  developed  for  each  of  these  performance  indicators.  In  addition  to  the  115 
observer-based  measures  developed  in  this  effort,  33  additional  system-based  measures  were 
defined  using  simulator  data  available  during  ATX.  Further  development  and  validation  is 
required  before  the  prototype  measures  can  be  incorporated  into  a  set  of  usable  training  tools. 

Utilization  and  Dissemination  of  Findings: 

Prototype  paper  versions  of  observer-based  measures  were  disseminated  to  several  Combat 
Aviation  Brigades  upon  request  to  assist  in  home-station  training.  Findings  were  briefed  to  the 
Director  of  Simulation  at  the  U.  S.  Army  Aviation  Center  of  Excellence,  20  January  2011. 
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Developing  Performance  Measures  For 
Army  Aviation  Collective  Training 


Introduction 


Background 

Previously,  collective  (i.e.,  unit  level)  aviation  training  was  accomplished  through  live 
field  exercises.  However,  for  many  reasons  (e.g.,  limited  resources,  and  lack  of  access  to 
suitable  practice  areas),  live  training  is  less  feasible  than  in  the  past.  A  response  to  these 
limitations  was  the  development  of  the  U.  S.  Army  Aviation  Warfighting  Simulation  Center 
(AWSC),  a  networked  training  system  located  at  Fort  Rucker,  Alabama.  The  AWSC  consists  of 
a  total  of  24  networked  cockpit  simulators  that  can  be  reconfigured  to  represent  the  Army’s  four 
currently  operational  combat  helicopters  (AH-64D  Apache,  CH-47D/F  Chinook,  OH-58D  Kiowa 
Warrior,  and  UH-60  A/L  Blackhawk).  The  AWSC  executes  tactical  missions  in  a  shared  virtual 
environment  consisting  of  a  highly  accurate  geospecific  terrain  database  with  constantly  updated 
cultural  features  (e.g.,  buildings  and  streets).  From  various  vantage  points  within  this  virtual 
environment  (e.g.,  battle  master’s  station;  stealth  platfonn),  data  on  the  position,  location,  and 
movement  of  entities,  including  the  aircraft  represented  by  the  training  devices,  can  be 
electronically  captured. 

Using  the  AWSC,  a  Combat  Aviation  Brigade  (CAB)  can  participate  in  a  collective 
Aviation  Tactical  Exercise  (ATX)  that  places  CAB  aircrews  and  battlestaff  in  a  common  virtual 
environment.  ATX  is  the  most  important  virtual  aviation  exercise  for  Army  aviation  CAB -level 
training,  and  it  consists  of  a  week-long  mission  readiness  exercise  prior  to  deployment  to  theater. 
As  a  result,  the  Army  has  a  heavy  investment  in  and  reliance  on  networked  training  devices  that 
operate  in  shared  virtual  environments  in  order  to  prepare  units  for  battle.  While  the  primary 
purpose  of  ATX  is  to  assess  the  readiness  of  battlestaff,  it  also  provides  an  opportunity  for 
feedback  on  the  readiness  of  aircrews.  Currently,  ATX  Observer  Controllers  (OC)  not  only 
provided  feedback  to  battlestaff  throughout  the  exercise,  but  also  to  aircrews  on  collective  task 
performance. 

Even  though  individual  aviation  tasks  are  generally  well  defined,  aviation  collective  tasks 
are  poorly  defined  as  broad  mission  segments  that  Army  Aviation  teams  must  accomplish 
(Cross,  Dohme,  &  Howse,  1998).  Army  aviation  collective  tasks  for  reconnaissance  and  attack 
operations  are  outlined  in  Army  Training  and  Evaluation  Program  (ARTEP)  manual  1-126 
(Department  of  the  Army,  2006)  and  refer  to  those  aviation  tasks  that  require  coordination 
between  one  aircraft  and  another,  coordination  between  an  aircraft  (or  flight  of  two  or  more 
aircraft)  and  a  tactical  command  element  (e.g.,  Brigade  Aviation  Element),  and  coordination 
between  an  aircraft  and  a  Ground  Commander.  For  example,  coordinating  and  adhering  to  flight 
formation  and  flight  duties,  deconflicting  airspace,  fulfilling  communication  requirements,  and 
applying  rules  of  engagement  (ROE)  are  all  types  of  aviation  collective  tasks.  However,  the 
requisite  underlying  knowledge  and  skills  that  support  aviation  collective  tasks  cannot  be 
inferred  from  such  broad  functions  within  those  tasks,  and  nor  from  task  descriptions  that  lack 
objective  performance  criteria.  Rather,  behaviorally-anchored  indicators  of  aviation  team 
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performance,  which  link  observable  behaviors  to  discrete  benchmarks,  should  be  used  to 
evaluate  performance  on  aviation  collective  tasks.  That  evaluation  can  illuminate  the  underlying 
knowledge  and  skills  necessary  for  aviation  collective  tasks. 

Training  research  (e.g.,  Salas,  Bowers,  &  Rhodenizer,  1998;  Stewart,  Dohme,  & 
Nullmeyer,  2002;  Stewart,  Johnson  &  Howse,  2007)  has  demonstrated  that  the  lack  of  clear 
performance  assessment  criteria  fails  to  fully  exploit  the  effectiveness  of  simulation-training 
events.  Moreover,  the  military  value  of  simulation-based  training,  such  as  ATX,  is  detennined 
by  perfonnance  improvement  of  participants  within  the  virtual-training  environment  (Bell  & 
Waag,  1998).  In  the  case  of  ATX,  there  is  a  need  to  develop  performance  criteria  on  aviation 
collective  tasks  in  order  to  assist  OCs  in  providing  feedback  to  aircrews  and  Leaders.  It  is  not 
enough  simply  to  identify  what  collective  tasks  aircrews  can  perfonn  at  the  end  of  ATX. 

Instead,  simulation-based  training  like  ATX  must  provide  opportunities  for  feedback  on  specific 
skills  and  for  correction  of  performance  in  order  to  improve  learning  (e.g.,  Bransford,  Brown,  & 
Cocking,  2000;  Ericsson,  Krampe,  &  Tesch-Romer,  1993).  Thus,  in  order  to  increase  the 
training  effectiveness  of  ATX,  there  is  a  need  (a)  to  identify  observable  indicators  that  define 
levels  of  performance  on  aviation  collective  tasks,  and  (b)  to  create  measures  that  assess  aviation 
collective  task  performance  during  ATX. 

The  sophistication  of  the  virtual-training  technology  supporting  ATX  stands  in  contrast  to 
the  way  in  which  collective  performance  is  measured.  Currently,  there  are  limited  systematic 
means  by  which  collective  performance  is  quantified  during  ATX.  Instead,  OCs  attempt  to 
capture  critical  incidents  that  illustrate  representative  performance  for  a  given  unit.  While  these 
critical  incidents  are  recorded  in  the  simulation  data  and  can  be  “replayed”  as  feedback,  defining 
critical  incidents  and  utilizing  available  simulator  data  to  illustrate  a  critical  incident  depends 
solely  on  the  unaided  ability  of  an  OC  to  notice  and  note  the  event.  By  contrast,  designing  and 
implementing  effective  performance  measures  usually  relies  on  a  variety  of  techniques  (e.g., 
system-based,  observer-based,  and  self-report)  to  fully  capture  perfonnance  (e.g.,  Campbell  & 
Fiske,  1959;  Jackson,  et  ah,  2008).  In  addition,  measures  of  collective  performance  should 
capture  both  outcomes  and  processes  of  the  collective  behavior  (Bell  &  Waag,  1998).  In  ATX, 
system-based  (i.e.,  simulator)  data  can  be  used  to  extract  measures  such  as  timing  of  events  or 
success  of  an  attack  while  observer-based  data  can  provide  insights  that  are  not  easily  obtained 
from  system-based  data  (e.g.,  communication  patterns  or  team  interactions),  and  self-report  data 
can  provide  information  on  cognitive  factors  that  are  not  easily  externally  observable  (e.g., 
workload,  situation  awareness).  Instead  of  relying  on  OC  observations  alone  to  capture 
collective  perfonnance,  the  integrated  use  of  multiple  types  of  measures  guided  by  training 
objectives  and  mission  scenarios  can  provide  a  comprehensive  representation  of  aviation 
collective  performance. 

Technical  Objectives  and  Scope  of  Research 

The  primary  objective  of  this  research  effort  was  to  develop  a  tool  that  could  assist  ATX 
OCs  to  assess  performance  on  aviation  collective  tasks.  This  tool  would  allow  OCs  to  provide 
behaviorally-based  feedback  to  aircrews  and  would  help  to  distinguish  high-performing  teams 
from  low-perfonning  teams.  Performance  results  from  across  training  units  could  then  be 
aggregated  to  provide  unit  leadership  with  a  “snapshot”  of  proficiency  on  aviation  collective 
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tasks,  resulting  ultimately  in  better-performing  teams.  To  achieve  this  objective,  a  set  of  critical 
aviation  collective  tasks  was  first  defined.  Next,  indicators  of  high  perfonnance  and  low 
performance  on  the  identified  collective  tasks  were  developed.  Finally,  measures  were 
developed  to  quantify  task  performance  and  to  develop  a  systematic  structure  for  assessing 
feedback.  Another  important  consideration  was  to  utilize  automated  simulator  data  to  measure 
collective  performance  whenever  possible.  Automating  the  measurement  process  could  augment 
observation-based  measures  or,  in  some  cases,  could  obviate  the  necessity  of  observational 
measurement.  In  this  research  effort,  the  objective  was  to  identify  and  define  both  observational 
and  automated  measures  that  could  eventually  be  implemented  in  data  collection  tools. 

It  is  important  to  note  that  for  the  purposes  of  this  research  the  type  of  aviation  collective 
tasks  was  intentionally  constrained.  The  Army’s  four  operational  helicopter  types  represent  four 
different  types  of  missions:  attack,  lift  (i.e.,  cargo),  scout-reconnaissance,  and  utility.  From  a 
tactical  standpoint,  attack  and  scout-reconnaissance  appear  to  be  the  most  demanding  missions 
because  these  missions  involve  interaction  with  hostile  forces  on  the  battlefield,  constant 
coordination  with  battlestaff  at  tactical  operations  centers  (TOCs)  and  Ground  Commanders,  and 
the  identification,  detection  and  engagement  of  targets.  In  short,  attack  and  scout-reconnaissance 
teams  are  the  most  likely  to  be  exposed  to  the  risks  inherent  in  combat.  For  these  reasons,  the 
current  research  effort  was  limited  to  collective  tasks  critical  to  perfonning  typical  missions  that 
Attack  Weapons  Teams  (AWT)  and  Scout  Weapons  Teams  (SWT)  train  and  experience  in 
combat. 


Method 

The  methodology  for  measure  development  combined  the  experiential  knowledge  base  of 
subject  matter  experts  with  established  psychometric  practices.  The  process  ensures  that  subject 
matter  experts  (SME)  work  collaboratively  with  scientists  to  reveal  insights  and  drive  the 
creation  of  measures  (e.g.,  Seibert,  Diedrich,  MacMillan,  &  Riccio,  2010).  This  methodology  is 
referred  to  as  COmpetency-based  Measures  for  Performance  Assessment  Systems 
(COMPASSsm).  The  COMPASS  process  was  initially  developed  to  assess  performance  of  a 
team  of  F-16  pilots  in  training  for  air-to-air  combat  in  a  high-fidelity  simulation  environment. 
(MacMillan,  Entin,  Morley,  &  Bennett,  in  press).  More  recently,  the  method  has  been  extended 
to  develop  observer-  and  system-based  measures  for  a  wide  range  of  applications  including  the 
Air  and  Space  Operations  Center’s  Dynamic  Targeting  Cell,  U.S.  Marine  Corps  Motorized 
Patrols,  U.S.  Navy  submarine  Fire  Control  Technicians,  and  U.S.  Army  Outcomes-Based 
Training  and  Education,  as  well  as  other  domains  (e.g.,  Jackson  et  ah,  2008;  Riccio,  Diedrich,  & 
Cortes,  2010). 

The  COMPASS  methodology  employs  an  iterative  series  of  three  workshops  with 
subject-matter  experts  to  develop  and  initially  validate  performance  measures.  The  COMPASS 
process  starts  with  identifying  key  training  objectives,  competencies,  and/or  selected  missions 
for  focus.  Using  these  items,  performance  measurement  requirements  are  elicited  from  SMEs  in 
the  first  workshop  in  the  form  of  Performance  Indicator  (Pis).  Pis  refer  to  observable  behaviors 
that  allow  an  individual  to  rate  the  quality  of  individual  or  team  performance.  In  the  second 
workshop,  more  detailed  information  is  gathered  for  each  PI  in  order  to  identify  a  range  of  likely 
and  desired  behaviors.  This  information  is  then  used  to  create  behaviorally-anchored 
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performance  measures  and/or  to  define  system-based  indications  of  performance.  The  goal  of 
the  third  workshop  is  to  conduct  a  detailed  review  and  to  modify  a  set  of  draft  performance 
measures.  As  part  of  this  detailed  review,  SMEs  confirm  the  relevance  of  each  measure  and 
ensure  that  each  performance  measure  appropriately  represents  the  behaviors  described  in  the  Pis 
derived  during  the  first  workshop. 

Participants 

For  the  current  research  effort,  the  COMPASS  methodology  was  applied  over  the  course 
of  three  small-group  sessions  (i.e.,  workshops)  with  SMEs  from  diverse  professional,  civilian, 
and  military  backgrounds.  The  heterogeneous  backgrounds  of  the  SMEs  ranged  from  military 
aviators  to  simulation  training  experts  and  software  engineers.  SMEs  represented  two  main 
organizations  of  the  U.S.  Army  Aviation  Center  of  Excellence:  the  Directorate  of  Simulation 
(DOS);  and  the  Training  and  Doctrine  Command  Capability  Manager  (TCM)  for 
Reconnaissance- Attack  (RA).  In  addition,  SMEs  were  recruited  from  the  Aviation  Captain’s 
Career  Course.  Across  the  workshops,  some  SMEs  participated  in  all  three  workshops,  whereas 
others  participated  in  only  one  workshop.  This  mix  of  participants  ensured  consideration  of  a 
variety  of  viewpoints. 

COMPASS  Workshop  One  took  place  on  22-23  June  2010  at  Fort  Rucker,  AL,  with  a 
group  of  participants  from  DOS  and  TCM-RA.  The  1 1  SME  participants  included  three 
experienced  active  duty  Kiowa  Warrior  (OH-58D)  pilots,  two  active  duty  Officers  who  were 
knowledgeable  on  ATX  operations  and  simulations,  three  retired  Army  aviators  with  current 
expertise  and  knowledge  of  Aviation  Combined  Arms  Tactical  Trainer,  Unmanned  Aircraft 
System  (UAS)  simulation,  simulation  and  training  operations,  and  three  additional  DOS 
personnel  with  experience  in  virtual  systems,  simulations,  and  Army  aviation  training. 

COMPASS  Workshop  Two  took  place  15-16  July  2010  at  Fort  Rucker,  with  additional 
follow-up  interviews  for  several  individuals  in  order  to  complete  data  collection  over  the 
subsequent  month.  Altogether  during  this  workshop  period,  nine  SMEs  from  Workshop  One  as 
well  as  six  new  SMEs  participated  in  the  process.  Of  the  new  SMEs,  four  were  current  students 
in  the  Aviation  Captain’s  Career  Course,  one  was  a  retired  Army  aviator  who  now  works  for 
DOS  along  with  several  of  our  other  workshop  participants,  and  one  was  an  active  duty  Army 
aviator  currently  assigned  to  DOS. 

COMPASS  Workshop  Three  took  place  on  26-27  October  2010,  also  at  Fort  Rucker. 
Similar  to  Workshops  One  and  Two,  the  SME  participants  included  ten  of  varying  backgrounds 
and  expertise.  Five  SMEs  had  participated  in  both  of  the  prior  workshops;  there  were  five  new 
workshop  participants.  Of  those  who  participated  in  Workshops  One  and  Two,  one  was  an 
experienced  active  duty  Kiowa  Warrior  pilot,  two  were  retired  Army  aviators  with  current 
expertise  and  knowledge  in  simulation  and  training,  and  two  were  DOS  personnel  with 
experience  in  virtual  systems,  simulations,  and  training  Army  aviation  collective  tasks.  Of  the 
new  participants,  three  were  active  duty  Kiowa  Warrior  pilots,  one  was  an  active  duty  Apache 
Longbow  (AH-64D)  pilot,  and  one  was  a  recently  retired  Kiowa  Warrior  pilot. 
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In  addition  to  individuals  participating  in  COMPASS  workshops  at  Fort  Rucker,  three 
Company  Commanders  within  a  CAB  were  interviewed  at  their  home  station  to  verify  collective 
training  needs  and  priorities.  All  three  Company  Commanders  had  operational  experience  in 
Iraq  and/or  Afghanistan,  and  each  was  preparing  for  deployment  under  a  different  task  force. 
Two  were  Kiowa  Warrior  pilots  and  one  was  a  Chinook  (CH-47)  pilot. 

Procedure 

COMPASS  Workshop  One.  The  goals  of  the  first  COMPASS  workshop  was  to 
identify  the  workflow  (i.e.,  flow  of  tasks  and  events  over  time)  for  collective  tasks  and 
interactions  performed  by  Army  aviation  aircrews  and  flights  in  attack/reconnaissance  missions 
and  to  derive  a  set  of  Pis  relevant  to  the  crews,  tasks,  and  mission  being  analyzed.  A  PI  is  an 
observable  behavior  that  allows  an  expert  (i.e.,  one  familiar  with  the  mission  objectives  and  task 
requirements)  to  recognize  whether  an  individual  or  team  is  performing  well  or  poorly.  During 
this  step  of  the  COMPASS  process,  it  was  critical  to  identify  observable  rather  than  inferred 
behaviors.  The  resulting  Pis  and  relevant  missions/tasks  provided  a  solid  basis  on  which  to 
develop  benchmarked  measures  that  were  less  sensitive  to  subjective  biases  and  more  reliable 
over  repeated  sessions.  In  addition,  the  Pis  provided  a  framework  on  which  to  develop  measures 
based  on  critical  decisions  and  events. 

Participants  focused  the  development  of  Pis  for  collective  tasks  within  a  flight,  within  an 
aircrew,  between  aircrews,  between  aircrews  and  TOCs,  and  between  aircrews  and  ground  forces 
in  an  attack/reconnaissance  scenario.  To  facilitate  the  development  of  relevant  Pis  during  the 
first  workshop,  a  hypothetical  mission  scenario  (see  Appendix  A)  was  developed  and  briefed. 
Several  factors  were  considered  in  the  development  of  this  scenario  in  order  to  provide  a 
complex,  realistic  mission  description.  First,  it  had  to  be  a  common  mission  for  an  AWT,  SWT, 
or  a  combination  of  the  two.  Second,  it  had  to  be  challenging  with  multiple  elements  involved 
during  the  mission.  Finally,  it  had  to  be  relevant  to  experiences  likely  to  occur  in  combat  for 
which  pilots  need  to  train.  Based  on  pilot  experiences  in  Iraq  and  Afghanistan  and  using 
tenninology  from  appropriate  ARTEP  manuals,  the  mission  scenario  was  developed  with 
combined  elements  of  Reconnaissance  and  Close  Combat  Attack  (CCA)  tasks  typical  of  current 
combat  missions.  Once  developed,  the  scenario  was  presented  to  the  Director  of  Simulation  and 
his  staff,  and  all  agreed  that  CCA  was  an  appropriate  collective  mission  to  use  for  this  effort. 

The  scenario  mimicked  those  currently  used  at  ATX,  and  the  mission  provided  a  framework  on 
which  to  identify  the  critical  events  and  decisions  that  needed  to  be  measured. 

COMPASS  Workshop  Two.  While  some  Pis  identified  in  Workshop  One  were  readily 
translated  into  performance  measures,  more  detailed  information  was  generally  required  in  order 
to  create  behaviorally-anchored  perfonnance  measures.  That  is,  for  a  given  PI,  the  specific 
behaviors  related  to  performing  poorly  or  perfonning  well  needed  to  be  detennined  in  order  to 
create  performance  measures  with  appropriate  rating  scales.  COMPASS  Workshop  Two, 
therefore,  focused  mostly  on  one-on-one  interviews  (one  to  three  hours  each)  to  discuss  the  Pis 
and  identify  explicit  behaviors  that  were  representative  of  good,  average,  and  poor  performance 
for  each  of  the  Pis.  Using  individual  interviews  was  thought  to  be  a  more  thorough  and  efficient 
method,  compared  to  group  sessions,  for  obtaining  detailed  information  required  for  the 
development  of  behaviorally-anchored  measures  and  scales. 
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During  the  interviews,  a  variety  of  questions  were  asked  to  obtain  infonnation  describing 
personnel  most  responsible  for  each  PI,  to  elicit  behavioral  anchors  relevant  to  each  of  the  Pis, 
and  to  determine — from  the  perspective  of  the  SMEs — the  appropriate  type  of  measures  to 
develop  for  each  PI  (i.e.,  systems-based  or  observer-based).  A  number  of  specific  questions 
were  also  posed  targeting  performance  parameters  for  the  development  of  system-based 
measures.  The  following  is  a  small  set  of  the  types  of  questions  asked  during  COMPASS 
Workshop  Two: 

•  What  might  a  member  of  the  flight  say  or  do  to  indicate  good/average/poor  performance 
for  this  PI? 

•  What  would  cause  a  person  to  do  well  or  poorly  at  this  PI? 

•  Does  this  person  interact  with  other  crewmembers,  the  ground,  or  their  TOC  for  this  PI? 

•  In  what  situations  during  this  step  of  the  mission  could  a  person  be  observed  perfonning 
well  or  poorly  for  this  PI? 

•  What  specific  tools/systems  do  help  accomplish  this  PI? 

•  What  simulator  data  may  be  published  that  can  be  used  to  assess  this  PI? 

Also  during  the  interviews,  two  to  three  individuals  from  the  research  team  took  detailed 
notes  and  logged  direct  quotes  as  often  as  possible.  Just  as  it  is  essential  for  multiple  note  takers 
in  a  single  interview,  it  is  essential  to  obtain  multiple  perspectives  on  each  PI.  A  single  SME 
may  only  be  able  to  provide  a  partial  description  of  the  situation,  or  may  provide  a  perspective 
not  shared  by  others.  By  recording  notes  from  several  researchers  on  perspectives  and 
descriptions  provided  by  a  number  of  SMEs  on  each  PI,  it  was  more  likely  that  the  resulting 
performance  measures  reflected  reality. 

The  information  gathered  during  the  Workshop  Two  interviews  was  used  in  post-workshop 
analysis  to  develop  tentative  sets  of  behaviorally-anchored  perfonnance  measures  and  system- 
based  measure  definitions.  This  process  involved  taking  each  PI  and  the  associated  notes 
obtained  in  Workshop  Two  and  creating  measures  using  behavioral  anchors  and/or  simulator 
data  that  define  good  and  poor  performance  for  that  PI.  Thus,  one  PI  could  have  one  or  more 
measures  associated  with  it,  and  these  measures  could  describe  observable  behaviors  for  either 
individual  roles  or  the  entire  flight  team.  Ultimately,  this  process  provided  analysts  with  a  set  of 
measures  that  could  be  used  together  or  in  separate  elements  depending  on  the  specific 
evaluation  criteria. 

Verification  of  critical  collective  tasks.  To  ensure  that  training  needs  and  priorities 
expressed  during  Workshop  One  and  Workshop  Two  were  consistent  with  current  needs  and 
priorities  of  CABs  in  theater  and  CABs  preparing  for  deployment  to  theater,  three  CAB 
Company  Commanders  were  interviewed  at  their  home  station.  During  these  interviews,  a  semi- 
structured  interview  format  was  employed  where  question  prompts  and  follow-up  questions  were 
proposed  and  open  discussion  of  topics  of  interest  was  encouraged.  A  sample  of  these  questions 
can  be  viewed  in  Appendix  B.  In  addition  to  tracking  operational  collective  training  priorities, 
supplemental  information  on  elements  of  good,  average,  and  poor  performance  for  collective 
task  performance  at  the  Company  and  aircrew  levels  for  topics  identified  during  the  interviews 
was  obtained. 
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COMPASS  Workshop  Three.  As  previously  mentioned,  the  COMPASS  process  is 
driven  by  SMEs  to  ensure  that  Pis  and  performance  measures  are  operationally  relevant,  as 
thorough  as  possible  given  the  mission  scenario,  and  appropriately  worded  using  the  experts’ 
language  and  terminology.  Therefore,  after  development  of  the  performance  measures,  the 
complete  set  of  measures  was  presented  to  SMEs  for  review  during  COMPASS  Workshop 
Three.  This  workshop  used  the  same  group  format  as  Workshop  One,  which  ensured  that  the 
final  set  of  performance  measures  was  understood  and  accepted  by  a  wide  range  of  users.  During 
this  workshop,  each  performance  measure  was  reviewed  with  respect  to  the  following  criteria: 

•  Relevance 

•  Observability 

•  Measure  type  (e.g.,  scale,  yes/no,  checkboxes;  system-based  vs.  observer) 

•  Measure  wording 

•  Scale  type 

•  Scale  wording 

In  real  time,  each  of  the  observer-based  and  system-based  performance  measures  was 
addressed  to  incorporate  the  inputs  of  SME  participants  with  respect  to  the  mentioned  criteria.  In 
addition,  SMEs  were  asked  if  there  were  additional  measures  that  needed  to  be  developed  (in  real 
time)  to  fill  any  gaps  in  the  measurement  framework  or  if  there  were  measures  that  needed  to  be 
removed  completely.  The  result  of  this  process  was  a  set  of  measures  that  were  developed, 
reviewed,  and  refined  by  a  wide  range  of  SMEs. 


Results 

Using  the  sample  mission  scenario  as  a  starting  point,  the  three  COMPASS  workshops 
leveraged  SME  knowledge  and  experience  to  identify  critical  skills  required  for  effective 
collective  performance  in  the  form  of  behaviorally-anchored  measures.  Behaviorally-based 
measures  are  systematic  descriptions  of  what  constitutes  good,  average,  or  poor  performance  in  a 
particular  job  or  task  and  the  knowledge  and  skills  needed  for  that  job  (MacMillan,  Garrity,  & 
Wiese,  2005).  The  results  of  this  process  yielded  Army  aviation  collective  task  performance 
measures  that  were: 

•  Behaviorally  anchored.  Behavioral  anchors  provide  raters  with  observable  features  of 
performance  that  observers  (or  a  measurement  software  system)  can  link  to  ratings  on  a 
scale. 

•  Designed  to  be  taken  at  critical  points  in  the  training  program.  Measures  taken  at 
specific  intervals  address  performance  at  critical  phases  in  the  exercise,  rather  than  as  an 
average  across  the  entire  exercise,  allowing  the  ratings  to  be  tied  to  specific  phases  in  the 
mission. 

•  Developed  to  evaluate  system-based  and  observer-based  behaviors.  Together, 
system-based  measures,  which  facilitate  automated  performance  feedback,  and  observer- 
based  measures,  which  facilitate  evaluative  feedback  that  systems  are  unable  to  capture, 
support  a  comprehensive  evaluation  of  collective  task  performance. 

•  Focused  on  aspects  of  performance  not  currently  standardized  across  OCs. 

Measures  guide  and  standardize  OC  observation,  facilitating  specific  behaviorally-based 
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feedback  to  each  unit  that  can  be  used  to  more  easily  identify  and  document  trends 
throughout  a  brigade  and  between  brigades. 

•  Useful  for  assessment  of  knowledge  and  skills  that  are  exercised  in  the  training 
environment.  Collectively,  the  items  are  designed  to  reflect  critical  objectives  from  the 
perspective  of  the  OCs  running  an  ATX. 

Taken  as  a  whole,  the  COMPASS  effort  yielded  three  products:  (1)  a  set  of  Pis 
representing  12  critical  mission  events  during  five  phases  of  the  exemplar  mission;  (2)  a  set  of 
observer-based  behavioral  measures  that  can  be  completed  manually  by  an  OC;  and  (3)  a  set  of 
system-based  behavioral  definitions  of  measures  that  can  guide  implementation  into 
measurement  software  which  can  collect  data  electronically  from  the  simulator  log.  The  PI  list 
and  two  sets  of  measures  reflect  the  anticipated  collective  tasks  performed  during  either 
preplanned  or  dynamically  re -tasked  aviation  missions.  The  PI  list,  observer-based  perfonnance 
measures,  and  system-based  performance  measures  can  be  viewed  in  Appendices  C,  D,  and  E 
respectively.  In  the  sections  that  follow,  the  specific  outcomes  of  each  step  throughout  this  effort 
are  described  in  more  detail. 

Outcomes  of  COMPASS  Workshop  One 

One  goal  of  Workshop  One  was  to  identify  Pis  that  represented  the  essential  elements  of 
an  example  mission.  In  general,  Pis  represent  critical  tasks  and  interactions  occurring  during  a 
mission  that  require  proper  execution  for  successful  mission  completion.  Pis  also  represent 
specific  opportunities  to  observe  measureable  behavior  during  the  course  of  a  mission  or  an 
operation  within  a  larger  mission.  Moreover,  Pis  represent  both  task  outcomes  and  the  processes 
used  to  achieve  a  given  outcome.  The  assessment  of  process  is  particularly  important  in 
consideration  of  collective  tasks  because  the  efficiency  of  team  interaction  is  a  hallmark  of  team 
performance  (e.g.,  Ilgen,  1999). 

The  general  format  of  a  PI  is  a  phrase  or  sentence  that  begins  with  an  action  verb  that 
focuses  on  an  observable  behavior.  For  example,  one  PI  reads:  “Confirm  target  with  appropriate 
technique  for  Ground  Commander  using  Standard  Operating  Procedures  (SOP).”  The  full  list  of 
Pis  was  formatted  in  a  spreadsheet  to  organize  the  Pis  and  to  show  the  hierarchical  dependencies 
among  Pis.  Accordingly,  the  PI  spreadsheet  numbered  each  PI  and  identified  the  personnel  most 
likely  to  exhibit  the  PI.  The  entire  PI  list  is  provided  in  Appendix  C  and  an  excerpt  appears  in 
Table  1.  This  list  was  also  used  to  organize  the  development  of  the  measures. 

The  PI  list  is  organized  according  to  an  operational  timeline  with  mission  phases  serving 
as  major  segments,  from  ‘Mission  Planning’  to  ‘Post  Flight  Tasks  and  After  Action  Review 
(AAR)’.  Each  PI  is  also  mapped  to  the  positions  (e.g.,  Fire  Support  Officer,  Battle  Captain),  or 
personnel  (e.g.,  aircrew,  aircrew  commander)  within  the  participating  unit  that  have  relevant 
actions  associated  with  the  PI.  Altogether,  Pis  were  developed  for  five  mission  phases  and 
further  broken  down  into  12  mission  events.  A  total  of  44  major  Pis  were  developed,  and  101 
additional  details  supporting  the  major  Pis  were  also  developed.  As  an  example,  in  the  sample 
exert  from  the  PI  list  in  Table  1 ,  items  7.1.1  and  7. 1 . 1 . 1  are  additional  supporting  Pis  to  the 
major  PI  7.1.  Similarly,  7.1,  7.2,  and  7.3  are  major  Pis  in  mission  event  7  ‘Apply  ROE’. 
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In  addition  to  representing  specific  observed  behaviors  and  interactions,  Pis  served  as  the 
context  from  which  performance  measures  were  developed.  As  an  example,  PI  6.5  is  located  in 
section  6  ‘Target  Acquisition’  and  is  the  last  step  before  section  7  ‘Apply  ROE’  (see  Table  1). 
This  PI  represents  actions  the  flight  is  perfonning  during  a  mission  but  prior  to  the  engagement 
of  a  target.  It  is  an  essential  step  in  the  target  acquisition  process  in  order  to  ensure  that 
subsequent  actions  (e.g.,  firing  on  the  target)  are  executed  properly  and  on  the  right  subject  (e.g., 
the  desired  target).  In  the  next  step  of  the  COMPASS  process,  each  PI  was  evaluated 
individually  to  obtain  acceptable  and  unacceptable  ranges  of  performance  on  associated  tasks. 


Table  1 

Sample  Excerpt  from  Performance  Indicators  List. 


Mission  Event 

PI  Number 

PI  Title 

Position 

Mission  Execution  Phase 

6  Target  Acquisition 
(In  parallel  with  on 
station  tasks) 

6.1 

Communicate  Last  Known  Position  and 

Description  of  Target 

Ground  Commander 

6.1.1 

Request  this  information  if  not  given  freely 

Air  Mission  Commander 

6.2 

Begin  Search  for  Target 

Flight  Team 

6.2.1 

Incorporate  the  ISR  Plan 

Flight  Team 

6.2.2 

Visual 

Flight  Team 

6.2.3 

Sensor 

Flight  Team 

6.2.3. 1 

Choose  proper  sensor  given  ambient  conditions 

Flight  Team 

6.2. 3. 2 

Share  sensor  feeds  if  required 

Flight  Team 

6.2.4 

Recognize  threats 

Flight  Team 

6.2.4. 1 

Utilize  Appropriate  Standoff  Distance 

Flight  Team 

6.3 

Announce  target  in  sight 

Flight  Team 

6.3.1 

Wingman  confirm  target 

Flight  Team 

6.4 

Communicate  Target  Acquisition  to  ground  forces 

Aircrew  and  Ground  Commander 

6.5 

Confirm  target  with  appropriate  marking  technique 
for  ground  commander  using  SOP 

Aircrew  and  Ground  Commander 

7  Apply  ROE 

7.1 

Confirm  ground  commanders  intent 

Ground  and  Air  Commander 

7.1.1 

If  apply  lethal,  determine  hostile  intent 

Ground  and  Air  Commander 

7.1. 1.1 

Ground  commander  or  AMC  must  confirm  hostile 
intent 

Ground  and  Air  Commander 

7.2 

Discuss  lethal  nonlethal  COAs 

Ground  and  Air  Commander 

7.3 

Discuss  proportionality 

Ground  and  Air  Commander 

7.3.1 

Desired  effect  accomplished  with  minimal 
collateral  damage 

Ground  and  Air  Commander 

Outcomes  of  COMPASS  Workshop  Two 

In  Workshop  Two,  each  PI  (i.e.  major  PI)  and  supporting  PI  (i.e.  additional  details 
supporting  a  major  PI)  was  discussed  with  SMEs  and  the  goal  was  to  gather  as  much  information 
as  possible  about  the  Pis  and  supporting  Pis.  At  the  conclusion  of  Workshop  Two,  notes  from 
all  interviews  were  compiled  and  organized  to  facilitate  meaningful  interpretation.  Once  the  full 
set  of  Workshop  Two  notes  was  organized,  each  PI  and  in  some  cases  supporting  PI,  was 
characterized  by  a  question  that  represented  a  behavior  amenable  to  an  observer-based  or  a 
system-based  measure.  Questions,  scale  types,  and  scale  anchors  for  each  PI  were  developed. 
Scale  types  were  determined  based  on  the  nature  of  the  question  and  the  available  information  to 
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assess  it.  If  a  task  or  procedure  was  so  simple  or  so  regimented  that  there  was  no  behavior 
between  right  and  wrong,  a  yes/no  scale  was  applied.  Other  tasks  reflected  a  set  of  regimented 
procedures  or  a  checklist  of  communications  or  procedures  that  must  be  followed  the  same  way 
every  time.  For  these  situations,  a  checklist  was  the  most  appropriate  means  of  assessing 
performance.  While  yes/no  and  checklist  questions  did  occur  on  occasion,  the  majority  of  items 
were  developed  into  Likert-type-scale  items  where  a  ‘1’  indicated  poor  behavior  and  a  ‘5’ 
indicated  the  best  possible  behavior. 

To  demonstrate  the  procedure  applied  in  the  development  of  performance  measures,  PI 

6.5,  ‘Confirm  target  with  appropriate  technique  for  Ground  Commander  using  SOP,’  can  serve 
as  an  example  (see  Figure  1).  A  review  of  sample  notes  compiled  during  Workshop  Two 
interviews  revealed  several  behaviors  that  reflected  poor,  average,  and  good  performance  on  PI 

6.5.  In  this  example,  the  notes  referred  to  understanding  how  to  discuss  the  target  and  confirm 
its  identity  within  the  flight  crew  as  well  as  with  the  ground  forces  using  proper  communications 
procedures  (e.g.,  follow  SOP).  These  notes  provided  some  general  descriptions  of  the 
procedures  as  well  as  examples  of  good,  average  and  poor  behavior.  During  measure 
development,  researchers  identified  key  words  or  phrases  that  illustrated  these  three  levels  of 
behavior.  These  notes  allowed  us  to  develop  appropriate  measures  with  behavioral  anchors  to 
compose  a  Likert  scale  item.  In  Figure  1,  the  key  words  and  phrases  identified  for  each 
performance  level  are  noted  by  thick  (good),  thin  (average),  and  dotted  (poor)  boxes.  Following 
the  identification  of  poor,  average,  and  good  behavior,  the  identified  key  words  and  phrases  were 
extracted  from  the  notes  and  formatted  into  a  draft  observer-based  performance  measure  (see 
Figure  2).  In  Figure  2,  N/A  equals  not  applicable;  NO  equals  not  observed. 

For  many  Pis,  notes  indicated  or  suggested  that  additional  measures  composed  of  system- 
based  data  could  be  developed.  System-based  measures  were  defined  based  on  data  understood 
to  be  on  the  simulator’s  events  database.  System-based  measures  can  provide  insight  into 
aspects  of  performance  that  are  difficult  for  humans  to  observe  or  to  reliably  report,  such  as 
coordinated  control  actions  and  aircraft  state.  In  contrast,  observer-based  measures  are  specific 
measures  rated  by  OCs  about  aspects  of  performance  that  are  more  difficult  to  assess  from 
available  system  data,  such  as  adherence  to  communications  standards.  However,  many  of  the 
actions  and  tasks  performed  by  the  aircrews  and  flights  involved  interaction  with  targeting 
systems,  sensors,  and  mission  control  software.  These  types  of  interactions  provided 
opportunities  to  develop  system-based  measures  using  data  already  being  published  in  simulator 
log  files.  These  system-based  measures  can  serve  as  either  alternatives  or  complements  to  the 
observer-based  measures.  In  the  case  of  PI  6.5,  a  draft  system-based  measure  definition  was  also 
composed  from  notes  gathered  in  Workshop  Two.  As  Table  2  shows,  the  draft  system-based 
measure  for  PI  6.5  reflects  key  actions  required  for  target  confirmation  that  involve  interaction 
with  the  rotorcraft’s  electronic  systems.  In  this  example,  the  system-based  measure  does  not 
look  exactly  like  its  corresponding  observer-based  measure.  However,  it  does  measure 
complementary  actions  indicated  by  SMEs  as  required  for  successfully  accomplishing  PI  6.5. 

As  part  of  the  system-based  measure  definition,  each  identified  measure  was  assigned  a 
status  indicating  the  likelihood  of  implementing  the  measure  definitions  in  current  system 
operations.  Detenninations  of  Likely,  Potential  and  Future  were  made  for  each  system-based 
measure  definition  based  on  an  assessment  of  current  simulator  operations.  Specifically, 


10 


6.5  Confirm  target  with  appropriate  technique  for  ground  commander  using  SOP 

Interview  Notes  1 

Knowing  SOP  and  be  able  to  discuss  target  in  accordance  with  SOPs  and  in  ways  that  ground 
forces  know  and  understand  (which  following  SOP  will  ensure) 

Average:  using  SOP  withf 


errors 


Poor:  disregard  for  established  procedures,  hesitation,  ifailjng  tp_co_nfirm_target  j 
Interview  Notes  2 

poor  -  doesn’t  use  appropriate  technique  for  marking  conditions  or  marks  wrong,  doesn’t  give 
ground  guy  [options  j 


avg  -  marks  target  with  appropriate  technique  ind  asks  ground  for  confirmation 
good  -  gets  ground  to  mark  as  well 

Interview  Notes  3 

Marking  target;[Taser7|Pire,  smoke,  ground  and  air  can  do  it.  The  gig  is  up  at  this  point. 

Great:  Selects  jnarking  approach;|Marks  and  acknowledges;|Use  of  brevity  codes;  Makes  a  call 
to  wing;  TaTT  units  ire  in  agreement 


Average:  Marks  and  acknowledges;  |Not  all  comms|between  groups  happen;  Average  guy  marks 
with  appropriate  and  asks  for  confirmation 

Poor:  Doesn't  use  [appropriate  marker^ [Doesn't  give  ground  guy  options;  Markslwrong  target  j 


[POOR!  I  AVERAGE 


GOOD 


Figure  1.  Example  notes  taken  from  Workshop  Two  for  Performance  Indicator  (PI)  6.5. 


6.5  Confirm  target  with  appropriate  marking  technique  for  Ground  Commander  using  SOP 

80.  Does  the  flight  mark  the  target  to  confirm  its  location? 


□  N/A 


1  2 
Flight  does[not  mark_] 
EPIr® ct Ja rg etjqr"u s es  the 
["incorrect  maker  [ 


Flightj  marks  target 


4 

5 

Flightidiscusses 

marking 

strategy  with 
mar 

ground;  | 

<s  target 

appropriately 

[POOR  1 


AVERAGE 


GOOD 


Figure  2.  Draft  observer-based  perfonnance  measure  from  Performance  Indicator  (PI)  6.5. 

distributed  interactive  simulation  (DIS)  data  log  files  from  previous  ATX  exercises  at  AWSC 
were  reviewed  and  analyzed.  The  purpose  of  this  assessment  was  to  detennine  the  likely  data 
generated  by  the  simulation  infrastructure  and  to  provide  a  first  pass  analysis  of  the  types  and 
quantity  of  data  that  is  available  over  this  infrastructure. 


11 


Table  2 

Draft  System-based  Measure  Definition  for  Performance  Indicator  (PI)  6.5:  Confirm  Target 
with  Appropriate  Marking  Technique  for  Ground  Commander  using  SOP. 


Category  of  Data 

Data  Required  for  System-based  Measurement 

Mission  Phase 

Mission  execution 

Mission  Event 

6  Target  Acquisition  (in  parallel  with  on-station  tasks) 

PI 

6.5  Confirm  target  with  appropriate  marking  techniques  using  SOP 

Status 

Likely 

Reason  for 
Classification 

Will  not  be  able  to  determine  if  they  used  the  appropriate  marking,  but  only 
if  they  correctly  used  the  chosen  marking. 

Performance 

Measure 

Does  the  flight  mark  the  correct  target?  Does  the  flight  use  the  appropriate 
technique  to  mark  the  target? 

Required  Data 
System 

Distributed  Interactive  Simulation  Network 

Required 
Simulation  Data 

Electromagnetic  Emission  Protocol  Data  Unit  (PDU);  Laser  designator; 
Position  of  target;  Position  of  laser  designator,  (could  also  be  gunfire  or 
rocket  fire  to  mark  target) 

Assessment 

Correct  or  incorrect  within  specified  acceptable  perfonnance  ranges 

Unit  of  Measure 

Feet;  Seconds 

Acceptable  Range 
of  Performance 

Exactly  on  target  for  Laser;  1 5  feet  for  rocket  or  gunfire 

Frequency  of 
Occurrence 

Once  when  target  is  marked 

Triggering  Event 

Engaging  the  designator 

Additional  Notes 

If  there  are  any  questions  regarding  target,  pilot  will  ask  ground  to  use 
smoke  or  gunfire  to  identify  target.  If  ground  is  already  engaging  target, 
clearance  of  fires  is  already  complete.  Will  use  laser  at  night  unless  IR 
laser  is  used.  IR  laser  requires  goggle  use  at  night.  If  a  second  type  of 
laser  is  emitted  for  aircraft  or  UAS,  that  laser  can  be  used  to  designate 
target.  Can  use  coded  laser  to  guide  weapon.  PDUs  tell  hit  or  miss  and 
why.  How  much  own  sensor  is  used  vs.  other  sensor  will  depend  on  units 
and  type  of  aircraft.  Smoke  during  the  day  is  good.  Gunfire  is  good 
because  clearance  of  fires  is  complete. 
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The  process  that  was  used  to  review  and  analyze  the  DIS  data  log  fdes  was  as  follows. 
First,  documentation  from  the  Institute  of  Electrical  and  Electronics  Engineers  Standards  for  DIS 
Application  Protocols  (Institute  of  Electrical  and  Electronics  Engineers,  1996)  and  the 
Simulation  Interoperability  Standards  Organization  Enumeration  and  Bit  Encoded  Values  for  use 
with  Protocols  for  DIS  Applications  (Simulation  Interoperability  Standards  Organization,  2006) 
were  reviewed  to  obtain  the  information-technology  protocols  required  for  DIS  and  the  specific 
numerical  values  and  associated  definitions  for  DIS  applications.  The  definitions  for  protocol 
data  units  (PDU)  were  also  obtained.  PDUs  are  data  messages  that  are  exchanged  on  a  network 
between  simulation  applications.  The  next  step  was  to  replay  data  from  an  ATX  event  and 
analyze  the  content  and  type  of  data  being  communicated  within  the  simulation  network.  The 
PDU  types  sent  over  the  simulation  environment  were  recorded  and  analyzed  at  a  field  and  data 
level,  as  well  as  with  respect  to  the  key  PDU  packets  identified  as  critical  to  the  system-based 
measures.  The  results  of  the  data  log  review  and  analysis  suggest  that  there  is  enough  data 
available  on  the  simulation  network  to  inform  a  variety  of  system-based  measures.  Possible 
system-based  measures  include  skills  beyond  the  reconnaissance-attack  mission  that  was  the 
subject  of  the  present  investigation.  Together,  the  results  of  this  PDU  type  and  field  analysis 
should  facilitate  the  implementation  of  system-based  measure  definitions  at  AWSC. 

The  analysis  of  DIS  log  files  and  the  PDUs  indicated  that  a  number  of  collective 
performance  measures  could  be  measured  with  system  data.  A  system-based  measure  was 
assigned  a  Likely  status  if  review  of  system  operations  suggested  the  required  simulator  data  and 
information  appears  to  be  available  in  current  simulator  log  files.  An  example  of  a  Likely 
measure  is  provided  in  PI  6.5  defined  in  Table  2.  A  system-based  measure  was  assigned  a 
Potential  status  if  the  review  of  system  operations  suggested  the  required  simulator  data  may  be 
available  but  it  was  not  clear  how  easily  the  data  could  be  obtained.  System-based  measures 
requiring  observer-based  measure(s)  as  triggering  events  were  also  given  Potential  status  because 
their  ability  to  assess  performance  hinges  on  implementation  of  observer-based  measures.  For  an 
example  of  a  Potential  system-based  measure,  see  ‘PI  3.3  Launch  Order’  in  Appendix  E.  This 
item  was  given  a  Potential  status  because  it  requires  a  comparison  of  the  reported  launch  order 
(to  be  obtained  through  observer-based  methods)  with  time  of  take-off  (to  be  obtained  through 
system-based  methods).  Finally,  a  system-based  measure  was  assigned  a  Future  status  if,  based 
on  current  simulator  operations,  it  does  not  appear  that  the  measure  can  be  implemented  (i.e., 
additional  simulator  functionality  is  needed).  For  an  example  of  a  Future  system-based  measure, 
see  ‘PI  1.1  Coordination  for  Brief  Preparation’  in  Appendix  E. 

During  the  post-Workshop  Two  measure  development  effort,  draft  performance  measures 
(observer-based  measures  and/or  system-based  measures  as  appropriate)  like  those  shown  in 
Figure  2  and  Table  2  were  developed  for  each  PI.  In  some  instances,  one  measure  was 
developed  for  each  PI.  In  other  cases  there  were  multiple  measures  for  one  PI  or  multiple  Pis 
covered  in  one  measure.  Where  infonnation  was  missing  or  confusing  in  notes,  comments  were 
made  to  prompt  discussion  for  clarification  during  Workshop  Three.  No  assumptions  were  made 
regarding  the  intent  of  a  SME’s  description  without  documentation  and  subsequent  verification 
of  these  assumptions.  At  the  end  of  the  measure  development  effort  between  Workshops  Two 
and  Three,  there  were  130  draft  observer-based  and  41  draft  system-based  measures. 
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Outcomes  of  Verification  of  Critical  Collective  Tasks 


During  this  step,  three  current  Army  aviation  company  commanders  preparing  for 
deployment  were  interviewed  to  ensure  the  training  needs  and  priorities  expressed  during 
Workshops  One  and  Two  were  consistent  with  current  needs  and  priorities  of  deploying  CABs. 
While  the  company  commanders  had  slightly  different  backgrounds  and  experiences,  similar 
insights  were  shared  regarding  training  needs  and  priorities.  Specifically,  training  needs  and 
priorities  similar  to  those  covered  in  the  PI  list,  and  identified  in  Cross  ,  et  al.  (1998)  surfaced. 
Tables  3  and  4  summarize  key  training  needs  that  surfaced  during  these  interviews.  For  each  key 
training  need,  relevant  Pis  and  specific  comments  from  the  aviators  interviewed  are  noted. 

Table  3 

Example  Performance  Indicators  (PI)  and  SME  Comments  Related  to  Understating  the 
Information  Needs  of  Friendly  Forces. 


Relevant  Pis 


SME  Commentary 


5.3.1  Establish  task  and  purpose  for 
aviation  assets 

7.1  Confinn  Ground  Commanders 
intent 


“Ability  to  meet  the  intent  of  ground  forces  is  key. 
Knowing  how  to  help  ground  forces  -  coaching 
ground  forces  about  your  capabilities  to  help.” 

“In  Afghanistan,  ground  forces  may  say  ‘we  are 
getting  this  report,  can  you  engage’  so  AMC  will 
have  to  decide  if  should  do  that.  Communication 
between  ground  forces  and  air  crew  is  critical  - 
MUST  seek  specific  clarification  from  ground  forces 
as  to  WHY  to  shoot  if  you  don’t  see  why  you  should” 


7. 1 . 1 . 1  If  apply  lethal,  determine  hostile 
intent  -  Ground  Commander  or  AMC 
must  confinn  hostile  intent 


“Ground  forces  may  clear  aircraft  hot  to  fire,  but  still 
falls  to  aircraft  if  there  are  any  questions  or  concerns 
over  why  they  shot. . .” 


In  order  to  verify  further  that  the  critical  aspects  of  collective  tasks  were  captured,  the 
developed  PI  list  was  compared  against  Anny  doctrine,  namely  ARTEP  1-126,  Mission  Plan  for 
the  Attack  Reconnaissance  Helicopter  Battalion/Squadron  (Department  of  Army,  2006).  A 
subset  of  tasks  in  the  ARTEP  was  selected  and  Pis  were  mapped  to  them  as  a  demonstration  of 
the  relevance  of  Pis  and  resulting  performance  measures  to  current  Army  doctrine.  As  CCA  was 
the  primary  focus  mission  for  perfonnance  measure  development,  the  ARTEP  and  Mission 
Training  Plans  (MTP)  were  used  as  guidelines  in  developing  the  scenario  and  guiding  the 
workshops.  As  workshop  participants  began  to  explore  the  scenario,  elements  for  a  Quick 
Reaction  Force  and  talk  of  tactical  movement  began  to  develop  in  relation  to  the  Pis  created.  As 
a  result,  it  was  anticipated  that  the  resulting  measures  would  provide  direct  links  to  a  number  of 
tasks  within  the  ARTEP  and  MTP  manuals.  Using  ARTEP  1-126,  this  expectation  was 
confirmed  by  mapping  Pis  to  collective  tasks  within  the  Training  and  Evaluation  Outlines 
(T&EO).  The  resulting  comparison  is  presented  in  Appendix  F.  The  goal  in  that  evaluation  was 
not  to  provide  another  training  guideline  but  to  enhance  the  current  processes  by  providing  a 
toolset  that  can  be  implemented  with  current  Anny  training  documents  (e.g.,  ARTEP,  MTPs,  and 
Units  Status  Reports).  Pis  and  resulting  performance  measures  show  a  level  of  abstraction  one 
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Table  4 

Example  Performance  Indicators  (PI)  and  SME  Comments  Related  to  Communicating 
Effectively  within  an  Aircrew,  between  Aircrews,  with  Ground  Forces,  and  with  the  Battalion 
TOC. 


Relevant  Pis 

SME  Commentary 

1 .3  Flight  team  and  aircrew  mission 
preparation 

“Good  team  (AMC)  will  disseminate  individual  tasks 
or  crew  tasks  within  the  SWT.  The  key  is  to  have  as 
detailed  a  plan  as  possible  before  you  take  off.” 

4.3  Monitor  and  acknowledge  updates 

“Good  team  knows  how  to  skin  the  cat  before  you  get 
out  there.  Knows  the  objective,  what  looking  for,  and 
how  to  conduct.  Checks  in  with  each  of  the  ground 
units  before  proceeding.  If  see  something  suspicious, 
lead  aircraft  communicates  this  to  trail  aircraft.  Left 
seat  trail  clarifies  any  gaps  because  he  does  the 
reporting.  Depending  on  what  you  are  seeing,  may 
have  the  AMC  call  the  ground  forces.” 

5.3.4  Discuss  plan  among  crew 

“Informal  conversation  between  lead  and  trail 
aircrafts” 

5.5  Develop  the  situation 

“Crews  talking  between  cockpits  to  find  the  required 
information  or  develop  the  situation.” 

5.6  Communicate  differences  in  pattern 
of  life  as  appropriate 

“Speaking  in  language  everyone  understands  (pro¬ 
words,  lighting  signals,  lingo)  -  communications.” 

7.5  Make  shoot/don’t  shoot  decision 

“If  anyone  in  an  aircrew  (flight  team)  has  questions 
about  if  should  engage  target,  may  be  best  to  not 
engage. . .  or  to  discuss  more.” 

down  from  the  T&EO  collective  task  list  from  the  ARTEP.  By  nesting  performance  measures  in 
the  ARTEP  tasks  where  appropriate,  Commanders  have  a  resource  that  provides  detailed 
information  as  to  “where”  a  unit  must  focus  to  successfully  accomplish  the  ARTEP  task.  This 
level  of  detail  will  allow  Commanders  to  focus  on  specific  collective  aviation  skills  and  provides 
a  more  concrete  evaluation  tool  in  relation  to  the  readiness  level  of  the  unit  to  perfonn  specific 
missions  such  as  CCA. 

Outcomes  of  COMPASS  Workshop  Three 

During  Workshop  Three,  draft  perfonnance  measures  were  reviewed,  edited  and 
finalized.  Conversations  in  the  workshop  were  focused  on  measure  and  scale  observability, 
clarity,  relevance,  importance,  and  wording.  All  required  revisions  were  made  to  measures  in 
real  time  during  the  workshop.  While  many  measures  were  modified  during  Workshop  Three,  a 
number  of  the  measures  were  deemed  appropriate  as  written  in  draft  form  and  thus  no  changes 
were  made.  SMEs  reviewed  all  measures  with  the  same  level  of  critique  but  detennined  that 


15 


some  appropriately  reflected  the  actions  of  the  PI  and  levels  of  performance  without 
modifications.  The  observer-based  performance  measure  for  PI  6.2.4. 1,  “Utilize  Appropriate 
Standoff  Distance,”  (question  number  74  in  Appendix  D),  for  example,  was  one  such  measure 
that  was  not  modified  during  Workshop  Three. 

The  observer-based  measures  for  PI  4.4.1,  “Loiter  or  Holding  Area”  (question  numbers 
35  and  36  in  Appendix  D)  is  an  example  of  a  measure  that  was  slightly  modified  during 
Workshop  Three.  Specifically,  workshop  participants  modified  the  language  in  the  scale 
anchors,  the  checklist  items,  and  decided  it  made  more  doctrinal  sense  to  split  the  question  into 
two  separate  questions,  one  for  loiter  area  and  one  for  holding  area  (the  original  item  was  one 
item  for  both  loiter  and  holding  areas).  In  accordance  with  Army  doctrine,  a  loiter  area  is  in  the 
air  whereas  a  holding  area  is  on  the  ground  or  hovering  over  a  specific  spot  on  the  ground. 

While  the  requirements  for  loitering  and  holding  areas  are  practically  the  same,  additional 
security  considerations  are  required  if  an  aircraft  is  to  land  in  a  holding  area.  Thus,  SMEs  felt  it 
was  more  appropriate  to  split  holding  and  loiter  areas  into  two  separate  measures. 

Summary  of  Products 

A  total  of  1 15  observer-based  measures  and  33  system-based  measure  definitions  were 
generated  reflecting  Pis  from  every  mission  phase.  Of  the  system-based  measure  definitions, 
there  were  12  measures  that  were  deemed  likely  to  be  implemented  given  current  system 
operations,  14  measures  that  may  potentially  be  able  to  be  implemented  given  current  system 
operations,  and  7  measures  that  could  be  implemented  if  system  operations  were  enhanced. 

When  considering  the  full  PI  list,  most  items  are  depicted  using  behaviorally  anchored  rating 
scales  that  describe  measurable  collective  task  behavior  at  the  novice,  average,  and  expert 
(coordinated  with  poor,  average  and  good)  levels.  Appendix  G  summarizes  each  type  of 
measure  developed  for  each  PI  in  this  effort  for  each  phase  of  the  mission.  In  this  summary  table, 
measures  are  grouped  by  PI  for  each  mission  phase,  with  corresponding  observer-based 
measures  categorized  according  to  number  and  system-based  measures  according  to 
developmental  status.  The  Likely,  Potential,  and  Future  distinctions  in  the  system-based  column 
indicate  the  implementation  status  for  the  measure  definition  corresponding  to  each  applicable 
PI.  The  final  list  of  1 15  observer-based  performance  measures  can  be  found  in  Appendix  D.  A 
summary  of  the  status  of  system-based  measures  can  be  viewed  in  Appendix  H,  and  the  final  list 
of  all  33  system-based  measures  can  be  found  in  Appendix  E. 

Together,  these  measures  operationalize  the  desired  behaviors  and  skills  of  a  set  of  Anny 
aviation  collective  tasks  in  terms  of  a  formative  assessment,  that  is,  an  assessment  that  is 
actionable  in  the  continual  improvement  of  training  and  education.  These  and  similarly  derived 
sets  of  performance  measures  can  be  implemented  during  ATX  as  well  as  at  other  deployment 
readiness  exercises.  Disciplined  use  of  these  measures  provides  the  opportunity  for  users  to 
become  increasingly  oriented  to  the  values  and  best  practices  within  Anny  aviation  collective 
task  performance. 
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Discussion  and  Recommendations 


The  intention  of  this  effort  was  to  provide  definitions  of  observer-based  and  system-based 
measures  of  Army  aviation  collective  task  perfonnance  in  attack-reconnaissance  missions  as 
currently  conducted  in  theater.  Once  developed,  tested,  and  implemented  into  user-friendly 
technologies,  the  resulting  performance  measurement  tools  could  be  used  to  enhance  the  training 
experience  of  aircrews  during  ATX.  This  research  effort  produced  (1)  a  set  of  Pis  representing 
critical  events  occurring  during  an  exemplar  mission,  (2)  a  set  of  observer-based  behavioral 
measures  that  can  be  manually  collected  by  an  OC,  and  (3)  a  set  of  system-based  behavioral 
measure  definitions  that  can  guide  implementation  into  measurement  software  to  electronically 
collect  data  directly  from  the  simulator  log.  These  prototype  tools  indicate  the  types  of  aviation 
collective  skills  that  should  be  evaluated  and  can  be  used  to  provide  constructive  feedback  on 
collective  outcomes  and  processes.  However,  additional  development,  testing,  and 
implementation  are  required  before  the  measures  can  be  effectively  applied  at  ATX. 

This  research  effort  was  successful  in  identifying  over  100  performance  indicators  of 
Anny  aviation  collective  skill.  All  of  these  performance  indicators  had  at  least  one  observer- 
based  measure  associated  with  it.  However,  only  33  possible  system-based  measures  were 
identified.  As  currently  defined,  system-based  measures  require  much  more  development  to  be 
usable  in  ATX  than  do  observer-based  measures  because  of  limitations  of  infonnation  in  the 
simulator  data  streams  and  of  challenges  associated  with  processing  available  data. 

In  order  to  use  the  developed  measures  during  ATX,  OCs  or  other  trainers  would  rate 
performance  in  real-time  during  regularly  scheduled  mission  events  using  observer-based 
performance  measures.  In  addition  to  the  collection  of  observable  measures,  system-based  data 
for  a  subset  of  measures  would  be  saved  for  later  processing  using  a  measurement  tool  that  meets 
the  specifications  previously  provided.  Following  completion  of  missions,  Companies  would 
receive  hotwashes  and  debriefs  as  usual  using  the  feedback  provided  by  the  measures.  The 
measures  can  also  be  used  to  detennine  the  extent  to  which  the  teams  (or  Companies)  performed 
well  on  the  key  training  objectives  the  Brigade  Commander  and  Company  Commanders 
indicated  as  high  priority.  For  example,  if  the  Brigade  Commander  wanted  a  special  focus  on 
ROE,  the  extent  to  which  the  teams  appropriately  handled  ROE  could  be  assessed. 

To  facilitate  use  in  ATX,  observer-based  measures  can  be  used  in  paper-based  fonnat  or 
be  implemented  into  a  hand-held  electronic  tool  (e.g.,  tablet  with  touch-screen  display)  and  used 
by  OCs  or  unit  Leaders  who  observe  the  training.  Implementation  of  the  observer-based 
measures  into  a  hand-held  electronic  tool  would  provide  observers  the  ability  to  quickly  and 
easily  select  only  those  measures  that  apply  to  a  given  training  event.  In  addition, 
implementation  into  an  electronic  tool  would  enable  recording  and  storage  of  the  measures  in  an 
electronic  format.  Likewise,  system-based  measures  can  be  implemented  in  a  software 
application  that  connects  directly  to  the  distributed  simulation  network  and  collects  data  required 
to  calculate  a  pre-defined  set  of  perfonnance  measures.  Having  all  of  the  performance- 
measurement  data  in  electronic  fonnat  will  make  it  easier  to  provide  feedback  during  AARs  and 
to  store  data  for  assessment  of  trends  over  time.  What  is  more,  performance  data  collected 
through  both  sources  (i.e.,  observer-based  and  system-based)  will  likely  be  needed  to  provide  a 
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clear  picture  of  performance  and  to  provide  feedback  to  both  participants  and  the  greater  training 
community. 

Further  development  of  the  measures  would  require  an  evaluation  of  validity  and  utility 
during  an  actual  training  event  (e.g.,  ATX).  Previously,  such  testing  has  enabled  substantial 
refinement  of  similar  types  of  measures  (e.g.,  Jackson  et  ah,  2008).  Testing  the  measures  in 
ATX  would  enable  refinement  of  wording,  identification  of  measurement  gaps,  facilitation  of 
workflow  for  ease  of  rating,  testing  of  inter-rater  reliability,  evaluation  of  response  variance,  and 
correlation  with  mission  outcomes.  However,  the  most  significant  measure-development  process 
would  involve  refining  the  system-based  measures  in  order  to  fit  them  into  a  usable  tool. 

There  are  a  number  of  requirements  a  system-measurement  tool  should  employ.  A  system 
measurement  tool  should  have  two  major  components:  The  first  component  is  a  data  adapter  or 
data  collection  program.  The  second  component  is  a  measurement  calculation  engine  that  uses 
the  data  collected  by  the  data  adapter  program.  The  data  adapter  program  should  be  capable  of 
collecting  data  generated  by  the  training  and  simulation  center’s  distributed-simulation  network. 
The  data  adapter  should  then  be  capable  of  storing  this  data  in  a  format  that  can  be  used  to 
calculate  performance  measures.  The  data  adapter  will  need  to  collect  data  from  simulation 
network  that  publishes  PDUs.  It  should  be  made  clear  that  such  a  tool  need  not  be  DIS  specific 
but  could  apply  to  other  simulation  systems  based  on  High  Level  Architecture  (HLA)  standard  in 
case  wider  dissemination  is  required.  The  measurement  calculation  engine  must  take  a  set  of 
measurement  definitions,  interpret  the  definitions,  and  perform  the  required  calculations  on  data 
being  collected  in  the  simulator  system.  The  calculation  engine  should  also  contain  the 
workflow,  or  the  business  logic,  of  the  assessment  calculations.  Hence,  producing  these  two 
major  components  for  the  data  available  at  ATX  requires  further  resources  and  development,  but 
the  results  of  the  current  research  (i.e.,  Pis  and  measure  definitions)  provide  a  framework  for 
doing  so. 

Even  though  the  measures  were  developed  for  training  conducted  at  an  ATX,  trainers  or 
unit  leaders  could  apply  the  observer-based  measures  to  any  home-station  training  exercise  the 
same  way  that  OCs  would  use  the  measures  at  ATX.  In  fact,  the  current  set  of  observer-based 
measures  was  disseminated  upon  request  to  CABs  who  participated  in  the  development  process. 
CAB  leaders  voiced  interest  in  incorporating  the  measures  in  their  training  programs. 
Additionally,  the  increasing  prevalence  of  home-station  simulation-based  trainers  such  as 
Aviation  Combined  Arms  Tactical  Trainer  (AVCATT)  and  the  Longbow  Crew  Trainer  (LCT) 
will  soon  make  system-based  measures  available  to  commanders  who  want  to  take  full 
advantage  of  such  training  resources. 

One  should  also  consider  the  “big  picture”  in  which  the  horizon  for  ATX  extends  far 
beyond  the  physical  confines  of  the  local  area  network  (LAN)  at  AWSC.  The  raison  d’etre  for 
DIS,  HLA,  and  other  networking  standards  is  that  participants  can  take  part  in  virtual  exercises 
from  various  remote  locations  over  a  wide-area  network  (WAN).  For  example,  a  Battalion 
Commander  at  home  station  should  be  able  to  link  AVCATT  and  LCT  to  AWSC  to  share  the 
same  virtual  environment.  An  Air  Force  Research  Laboratory  research  effort  (Portrey,  Keck,  & 
Schreiber,  2006)  summarizes  the  development,  evaluation,  and  architecture  of  system-based 
performance  measures  for  military  aviation  training  in  both  LAN  and  WAN  environments. 
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Portrey,  et  al.  (2006)  discuss  the  challenges  faced  in  developing  and  validating  ,  and 
demonstrating  system-based  perfonnance  measurement  architectures  appropriate  to  complex, 
distributed  virtual  environments  which  involve  disparate  simulator  systems,  characteristic  of 
joint/coalition  virtual  exercises.  Because  it  is  likely  that  Anny  Aviation  will  increasingly 
participate  in  joint  virtual  training  exercises  in  the  future,  collaboration  across  services  is 
necessary  if  the  systems  and  entities  on  the  WAN  are  to  interact  and  function  according  to  the 
same  criteria. 

In  summary,  the  results  of  this  research  provided  both  an  initial  understanding  of  the 
team  processes  required  for  successful  Anny  aviation  collective-task  perfonnance  as  well  as  a 
first  step  toward  the  measurement  of  aviation  collective-task  performance.  Collectively,  the 
developed  measures  show  promise  as  tools  to  provide  formative  feedback  to  guide  learning 
during  training.  As  constructed,  the  measures  provide  descriptions  of  observed  and  expert 
performance  and,  as  such,  the  measures  can  be  used  to  illustrate  desired  performance  across 
various  elements  of  collective  tasks.  Moreover,  in  addition  to  providing  individual  and  team 
feedback  during  training,  the  measures  could  also  facilitate  the  analysis  of  performance  trends 
over  time  and  within  and  across  units.  Analysis  of  performance  trends  could  identify  more 
global  perfonnance  improvements  and  persistent  training  challenges  at  aircrew,  flight,  Company, 
and  Battalion  levels.  Once  incorporated  into  appropriate  data  collection  tools,  system-based  and 
observer-based  measures  of  collective  task  performance  have  the  potential  to  improve  the 
Army’s  ability  to  maximize  effectiveness  of  training  time  for  mission  critical  tasks,  to  assess  the 
effectiveness  of  ATX  in  preparing  CABs  for  deployment,  and  to  improve  unit  perfonnance  by 
providing  specific,  constructive  feedback  to  CABs. 
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Acronyms 


ARTEP 

ATX 

AWSC 

AWT 

Army  Training  and  Evaluation  Program 

Aviation  Tactical  Exercise 

Aviation  Warfighting  Simulation  Center 

Attack  Weapons  Team 

CAB 

CCA 

COMPASS 

Combat  Aviation  Brigade 

Close  Combat  Attack 

COmpetency-based  Measures  for  Performance  Assessment  Systems 

DIS 

DOS 

Distributed  Interactive  Simulation 

Directorate  of  Simulation 

HLA 

High  Level  Architecture 

LAN 

Local  Area  Network 

MTP 

Mission  Training  Plan 

OC 

Observer  Controller 

PI 

PDU 

Perfonnance  Indicator 

Protocol  Data  Unit 

RA 

ROE 

Reconnaissance  Attack 

Rules  of  Engagement 

SME 

SOP 

SWT 

Subject  Matter  Expert 

Standard  Operating  Procedure 

Scout  Weapons  Team 

T&EO 

TCM 

TOC 

Training  and  Evaluation  Outlines 

Training  and  Doctrine  Command  Capability  Manager 

Tactical  Operations  Center 

UAS 

Unmanned  Aircraft  System 

WAN 

Wide  Area  Network 
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APPENDIX  A 
MISSION  SCENARIO 


A-l 


Collective  Tasks  under  Mission  -  Conduct  attack  reconnaissance  operations: 

Bn  Msn  01-1-5127  Conduct  Aerial  Close  Combat  Attack  Operations  p5-6 1(  147) 

S3  Section  01-1-5140  Plan  Aerial  Close  Combat  Attack  Operations  p5-68(154) 

Co  Msn  01-2-5199  Perform  Aerial  Close  Combat  Attack  Operations  p5- 179(265) 

References  -  FM  3-04.126,  FM-100,  FM  3-04.111,  FM  3-90,  FM  3-100.12,  FM  7-0,  FM  7-1,  TC 
1-210,  TC  1-248,  TC  1-251 

1 .  B/4-3 1  is  an  infantry  company  that  is  conducting  presence  patrols  in  the  southern  part  of 
Humdalia  valley.  They  are  using  a  Raven  UAS  to  screen  near  a  village  to  provide  the  company 
with  early  warning  as  they  search  for  weapons  caches  in  the  valley.  There  are  infantry  companies 
on  patrol  in  the  vicinity  but  none  directly  in  the  village.  The  Raven  feed  shows  a  possible 
technical  vehicle  that  the  infantry  battalion  has  been  looking  for  based  on  intel  from  a  HUMINT 
source.  The  truck  is  headed  in  an  east  to  west  direction  through  the  winding  village  streets. 

2.  4-3 1IN  battalion  battle  captain  requests  aviation  attack  assets  to  come  ‘on  station’  to  help 
track  and  identify  the  possible  gun  truck.  5-5  ARB  battle  captain  receives  the  infantry  company’s 
call  sign  and  freq  as  well  as  a  description  of  the  target,  its  location,  and  enemy  activity  in  the 
area.  The  battle  captain  then  calls  the  TOC  together  to  perform  a  quick  mission  analysis.  The  S2 
provides  the  enemy  situation  from  both  the  air  and  ground  perspectives.  The  battle  captain 
provides  analysis  on  the  amount  of  station  time  required  to  track  and  find  the  target,  type  of 
munitions  needed  to  destroy  the  target,  and  crew  experience.  Weighing  the  importance  of  the 
mission,  risk  analysis,  and  time  to  accomplish,  the  battle  captain  reviews  available  apache  teams 
(not  assigned  a  specific  mission)  and  assigns  the  mission  to  an  apache  team.  The  aviation  battle 
captain  approves  the  request  and  notifies  the  4-3 1IN  battle  captain  the  call  sign,  freq,  and  ETA 
for  the  apache  team.  The  5-5  ARB  battle  captain  also  de-conflicts  the  airspace  asking  that  the 
Raven  UAS  stay  to  the  north  above  grid  line  55  and  stay  at  or  below  500’  AGL. 

01-1-5410  Plan  Aerial  Close  Combat  Attack  Operations  Task  1-6 

3.  A  team  of  AH-64s  is  sent  to  the  B/4-3  IN  AO.  The  team  receives  a  call  sign,  grid,  freq, 
description  of  the  vehicle,  and  a  direction  of  travel  from  the  5-5  ARB  TOC.  The  team  also  is 
given  the  location  and  altitude  of  the  Raven.  Finally,  the  aviation  battle  captain  provides  the  team 
with  an  enemy  situation  from  both  the  air  threat  and  ground  threat  perspectives.  The  AH-64 
team,  call  sign  Killer  11,  acknowledges  the  brief  and  departs  for  the  B/4-3  IN  AO. 

01-1-5410  Plan  Aerial  Close  Combat  Attack  Operations  Task  1-6 
01-1-5199  Perfonn  Aerial  Close  Combat  Attack  Operations  Task  1-3 


A-2 


4.  Killer  1 1  approaches  the  B/4-3 1  IN  AO  and  checks  in  with  the  infantry  battalion  providing 
call  sign,  team  composition,  estimated  time  enroute,  weapons  configuration,  and  time  on  station. 
They  receive  a  quick  update  on  the  target,  location  of  the  Raven,  and  enemy  situation. 

01-1-5199  Perform  Aerial  Close  Combat  Attack  Operations  Task  3-4 

5.  The  infantry  battle  captain  then  pushes  Killer  1 1  to  the  infantry  company’s  net.  The  team 
contacts  the  infantry  company  and  calls  the  5-5  ARB  battle  captain  to  acknowledge  he  is  in  the 
area  and  has  made  contact  with  the  infantry  battalion. 

01-1-5199  Perfonn  Aerial  Close  Combat  Attack  Operations  Task  5 

6.  Killer  1 1  begins  reconnaissance  along  the  route  from  the  last  known  location.  They  maintain 
radio  comms  with  the  infantry  battalion  and  company  to  receive  updates  until  they  find  the 
target.  Killer  1 1  positively  identifies  the  vehicle  as  a  white  gun  truck  with  a  darker  color  door  on 
the  right  side.  The  vehicle  is  stopped  in  an  open  area  within  the  city  limits.  A  heavy  machine  gun 
is  identified  in  the  bed  of  the  truck  with  no  gunner  present.  The  vehicle  is  stationary  and  parked 
near  a  wall  that  separates  the  open  area  from  a  section  of  houses  along  a  main  thoroughfare.  The 
vehicle  is  running  and  the  driver  is  inside  the  vehicle. 

01-2-5189  Perfonn  Aerial  Area  Reconnaissance  Operations 

7.  Killer  1 1  runs  through  the  clearance  of  fires  talking  to  the  infantry  company  and  aviation 
battle  captain  to  ensure  there  are  no  friendlies  in  the  area,  they  have  the  correct  target  (not  an  IP 
vehicle),  collateral  damage  is  at  a  minimum,  and  the  ROE  is  appropriately  used  to  destroy  the 
gun  truck.  The  aviation  battle  captain  and  infantry  company  confirm  the  identification  of  the 
correct  vehicle  and  that  there  are  no  coalition  forces  in  the  vicinity. 

01-1-5199  Perfonn  Aerial  Close  Combat  Attack  Operations  Task  6 

8.  Killer  1 1  selects  the  30mm  cannon  and  destroys  the  vehicle.  It  reports  BDA  to  the  infantry 
company  (who  relays  to  the  infantry  battalion  battle  captain)  and  the  aviation  battalion  battle 
captain. 

01-1-5199  Perfonn  Aerial  Close  Combat  Attack  Operations  Task  8-9 
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APPENDIX  B 

SAMPLE  HOME  STATION  INTERVIEW  QUESTIONS 
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INTERVIEWEE  NAME  &  RANK: 

INTERVIEWER  NAME: 

DATE  AND  TIME: 

BACKGROUND  INFORMATION 

1 .  Purpose  of  this  project 

a.  This  work  is  funded  by  the  Anny  Research  Institute. 

b.  Our  primary  focus  is  on  ATX  at  the  U.  S.  Army  Aviation  Warfighting  Simulation 
Center  at  Fort  Rucker,  but  we  expect  our  measures  to  generalize  to  collective  tasks 
outside  of  ATX  as  well. 

c.  The  primary  goal  of  this  project  is  to  develop  measures  of  collective  tasks 
performance  for  tasks  between  aircrews  of  the  same  flight,  among  aircrews  in  the 
same  mission,  between  aircrews  and  ground  forces,  and  between  aircrews  and  their 
controlling  HQs  (TOCs). 

2.  Introductions 

a.  Troy 

b.  Melinda 

c.  Interviewee 

GENERAL  EXPERIENCE  WITH  COLLECTIVE  TASKS 

3 .  In  your  experience  and  opinion,  what  characteristics  describe  a  good  performing  air  team?  A 
good  performing  aviation  company?  A  good  perfonning  Battalion?  A  good  perfonning 
Combat  Aviation  Brigade?  What  about  a  bad  one  (Air  team,  Company,  Battalion,  Brigade)? 

4.  How  do  you  prepare  for  collective  missions  as  an  Air  team?  As  a  Company?  As  a  Battalion? 
As  a  Combat  Aviation  Brigade? 

5.  What  kinds  of  experiences  are  helpful  for  developing  as  an  air  team?  (in  aircraft  or  not,  in 
theater  or  exercise) 

6.  What  kinds  of  missions  might  you  encounter  that  require  high  levels  of  coordination  between 
air  teams? 

a.  What  about  between  your  air  team  another  aircraft  part  of  the  same  mission? 

b.  What  about  between  the  Battalion  TOC  and  aviation  teams? 

c.  What  about  between  air  teams  and  ground  forces? 

d.  What  about  between  air  teams  and  brigade  level  folks? 

7.  Besides  ATX  at  AWSC,  what  does  your  Company,  Battalion,  and  Brigade  do  to  train 
collective  tasks?  Are  there  other  TADSS  you  are  using? 

8.  For  your  upcoming  deployment,  what  kinds  of  collective  tasks  does  the  10th  CAB  anticipate 
they  will  need  to  perform  on  a  regular  basis? 

a.  At  the  Air  Team  level? 

b.  At  the  Company  level? 

c.  At  the  Battalion  level? 

d.  At  the  Brigade  level? 

9.  Of  these,  which  collective  tasks  do  you  consider  most  challenging? 

a.  Is  it  the  same  at  each  level  (air  team,  company,  battalion,  etc)  or  do  the  most 
challenging  collective  tasks  vary? 


B-2 


10.  Again,  of  those  you  listed,  which  collective  tasks  do  you  consider  easiest  to  perfonn? 

a.  Is  it  the  same  at  each  level  (air  team,  company,  battalion,  etc)  or  do  the  most 
challenging  collective  tasks  vary? 

1 1 .  Again,  are  there  any  that  are  more  critical  to  performing  well  as  a  CAB  than  others? 

a.  Is  it  the  same  at  each  level  (air  team,  company,  battalion,  etc)  or  do  the  most 
challenging  collective  tasks  vary? 

COLLECTIVE  TASKS  REQUIRED  IN  YOUR  LAST  DEPLOYMENT 

12.  When  was  your  last  deployment  as  part  of  an  aviation  brigade?  What  was  your  primarily 
role  during  your  deployment? 

13.  If  recently  deployed: 

a.  Based  on  your  most  recent  deployment,  how  did  your  unit  perfonn  on  the  specific 
areas  you  targeted  in  ATX? 

b.  Were  there  areas  you  knew  would  be  challenging  (esp.  collective  tasks)? 

c.  Were  there  collective  task  performance  issues  that  came  up  that  you  were  not 
expecting? 

d.  If  you  could  re-do  your  ATX  /  AWSC  exercise,  would  you  focus  on  any  additional 
collective  tasks? 

EXPERIENCE  WITH  ATX 

14.  How  much  experience  have  you  had  attending  or  leading  ATX?  Can  you  describe  your 
involvement?  What  role(s)  did  you  fill?  (i.e.  flight  lead,  AMC,  battle  captain,  etc.) 

15.  Have  you  been  to  the  AWSC  for  ATX?  What  was  your  experience  like? 

a.  How  useful  did  you  find  the  training  personally  as  an  aviator? 

b.  How  useful  did  you  find  the  training  for  your  company? 

c.  How  useful  did  you  find  it  for  training  at  the  battalion  and/or  brigade  levels? 

16.  What  areas  of  collective  task  performance  did  these  exercises  help  you  to  perform  better? 
(i.e.  communication,  identifying  what  you  know  and  do  not  know,  etc.) 

a.  What  about  your  company?  Battalion?  Aviation  brigade? 

17.  For  your  upcoming  exercise,  do  you  have  some  target  areas  for  collective  task  training? 

a.  What  are  areas  do  you  hope  to  work  on  individually  at  ATX?  What  about  as 
Company?  Battalion?  Aviation  brigade? 

b.  How  has  your  unit  been  preparing  for  ATX,  or  how  will  you  prepare? 

18.  What  type  of  feedback  is  helpful  for  training  collective  tasks  at  the  Air  Team,  Company, 
Battalion,  and  Brigade  levels? 

a.  If  you  were  in  an  exercise  and  you  were  doing  something  wrong  (as  an  individual), 
how  and  when  would  you  want  feedback  on  your  perfonnance? 

b.  Asa  team,  how  and  when  would  you  want  feedback  on  our  performance? 

c.  What  format  would  you  prefer  feedback  to  be  in  (verbal,  written,  etc.)? 

d.  Have  you  ever  had  a  hot  wash  that  you  will  never  forget?  Can  you  tell  me  about  it? 
Why  was  it  so  memorable  -  what  did  you  leam? 

e.  What  about  an  AAR? 

f.  At  an  event  like  the  ATX,  how  do  you  know  when  your  team,  company,  battalion, 
and  brigade  is  performing  well?  How  do  you  know  when  you  are  not? 
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19.  We  are  developing  measures  to  assess  collective  task  perfonnance  primarily  for  use  during 
ATX  at  AWSC.  What  do  you  believe  would  be  the  best  place  (hot  wash,  AAR?)  for 
collective  performance  results  to  be  presented?  To  whom  should  they  be  presented?  In  what 
format  (report  card,  computer  printout,  etc.)? 

a.  Would  you  want  to  just  know  how  you  did  or  would  it  be  helpful  to  also  know  how 
you  could  improve  (even  if  you  did  okay)? 

b.  What  level  (individual,  air  team,  company,  battalion)  would  be  appropriate  to  assess 
collective  task  performance  for  it  to  actually  change  future  performance? 

c.  In  general,  once  you  get  feedback  either  at  the  individual  or  more  so  at  the  collective 
level,  how  do  you  implement  feedback  to  ensure  improvement  the  next  time? 

ADDITIONAL  QUESTIONS  OR  COMMENTS 

20.  Do  you  have  any  additional  comments  for  us  about  your  experience  with  Collective  Task 
performance,  either  in  exercises  or  in  theater? 

2 1 .  May  we  contact  you  with  brief  (15  min)  additional  questions  after  your  unit  goes  through 
ATX?  If  there  is  time  while  we  are  on  site  (we  will  not  interfere  in  any  with  your  ATX!), 
would  you  be  willing  to  meet  with  us  again  for  15  minutes  or  so? 

a.  If  yes,  can  you  provide  us  with  your  email  and  phone  contact  information? 
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APPENDIX  C 

PERFORMANCE  INDICATOR  LIST 
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Mission  Event 

PI  Number 

PI  Title 

Position 

Mission  Planning  Phase 

1  Pre-Mission 

Planning 

1.1 

Coordination  for  Brief  Preparation 

Battalion  Staff 

1.1.1 

Coordinate  with  Adjacent,  Higher,  and  Lower  Units 

Staff  Counterparts,  LNO 

1.1.2 

Adherence  to  SOP  Battle  Rhythm 

Battalion  Staff,  Brigade  Staff 

1.2 

Operation  Summary  (Past,  Current,  Future) 

S3  Section 

1.3 

Flight  Team  and  Aircrew  Mission  Preparation 

Air  Mission  Commander 

1.3.1 

Flight  Team  Briefs 

Air  Mission  Commander 

1.3.2 

Establish  weapons  release  authority 

Air  Mission  Commander 

1.3.3 

Aircrew  Briefs 

Air  Mission  Commander 

2  Mission  Analysis 

2.1 

Flight  Team  Receipt  of  WARNO  and  Parallel 
Mission  Planning 

Flight  Team  (Air  Mission  Commander) 

2.2 

Mission  Coordination 

Battalion  Staff  and  Flight  Crews 

2.2.1 

Airspace 

Battalion  Staff  and  Flight  Crews 

2.2.2 

Coalition  Forces 

Battalion  Staff  and  Flight  Crews 

2.2.3 

Obtain  UAS  Feeds  (if  possible) 

Battalion  Staff  and  Flight  Crews 

2.2.4 

Update  Friendly  Situation 

Battalion  Staff  and  Flight  Crews 

2.2.5 

Verify  Communication  Frequencies 

Battalion  Staff  and  Flight  Crews 

2.2.6 

Verify  Call  Signs 

Battalion  Staff  and  Flight  Crews 

2.2.7 

Verify  Grid  Locations 

Battalion  Staff  and  Flight  Crews 

2.2.8 

Threat  Update 

Battalion  Staff  and  Flight  Crews 

2.2.9 

React  to  threat  update/change  COA 

Battalion  Staff  and  Flight  Crews 

(continued) 
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Mission  Event 

PI  Number 

PI  Title 

Position 

3  Task  Quick 

Reaction  Force 
(QRF) 

3.1 

Attend  Final  Mission  Brief  as  required 

Air  Mission  Commander 

3.1.1 

Final  Adjustments  to  Plan 

Flight  Team 

3.1.2 

Report  changes  to  aircrew 

Flight  Team 

3.2 

Request  SITREP  on  Net  (prior  to  launch) 

Air  Mission  Commander 

3.3 

Eaunch  Order 

Battle  Captain 

Enroute  Phase 

4  Enroute 

4.1 

Call  Off  to  Battalion  TOC 

Air  Mission  Commander 

4.1.1 

Battalion  Staff  log  the  flight  off  and  acknowledge 

Brigade  Battalion  Battle  Captain 

4.1.2 

Report  to  higher  Flight  Team  Departure 

Brigade  Battalion  Battle  Captain 

4.1.3 

Report  to  ground  forces  that  Flight  Team  is  Enroute 

Brigade  Battalion  Battle  Captain 

4.2 

Aircrew  to  deconflict  airspace  as  required  (air 

Flight  Lead 

4.3 

traffic  services) 

Monitor  and  Acknowledge  Updates 

As  assigned  according  to  SOP 

4.4 

Coordinate  Team  Tactics  with  wingman 

Flight  Lead  et  al. 

4.4.1 

Loiter  or  Holding  Area 

Flight  Lead  et  al. 

4.4.2 

Deconfliction  Measures 

Flight  Lead  et  al. 

4.4.3 

Delegation  of  coordination  and  flight  related  duties 

Air  Mission  Commander 

4.5 

(e.g.  Communications) 

Adherence  to  the  SOP 

Air  Mission  Commander 

4.5.1 

Formation 

Air  Mission  Commander 

4.5.2 

Flight  Duties 

Air  Mission  Commander 

4.5.3 

Communication  Protocol 

Air  Mission  Commander 

4.5.4 

SOP  Driven  Communication  (FARM  etc.) 

Air  Mission  Commander 

4.5.5 

Tactics 

Air  Mission  Commander 

(continued) 
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Mission  Event 

PI  Number 

PI  Title 

Position 

4.6 

Execute  Air  Ground  Integration  Checklist 

Aircrew  to  Ground  Commander 

4.6.1 

Check-in  with  Ground 

Battalion  and  Company  Nets 

4.6.1. 1 

Number  of  Aircraft 

Battalion  and  Company  Nets 

4.6. 1.2 

Number  and  Type  of  Weapons  System  Available 

Battalion  and  Company  Nets 

4.6. 1.3 

Station  Time 

Battalion  and  Company  Nets 

4.6. 1.4 

Request  SITREP 

Battalion  and  Company  Nets 

4.6.2 

Receive  SITREP  from  Ground 

Ground  Commander  to  Aircrew 

4.6.2. 1 

Frontline  Trace  and  unit  composition 

Ground  Commander  to  Aircrew 

4. 6. 2.2 

Markings 

Ground  Commander  to  Aircrew 

4. 6. 2. 3 

Updates  to  the  situation 

Ground  Commander  to  Aircrew 

4. 6. 2.4 

Obtain  UAS  feed  (if/when  possible) 

Ground  Commander  to  Aircrew 

4. 6. 2.4.1 

Goal  is  to  obtain  as  soon  as  possible  in  mission 
thread 

Ground  Commander  to  Aircrew 

4. 6. 2. 5 

Immediate  Airspace  Deconfliction  and/or 

Avoidance  Measures 

Ground  Commander  to  Aircrew 

Mission  Execution  Phase 

5  Arrive  on  Station  - 
Update  Situational 
Awareness 

5.1 

Communicate  Arrival  On  Station 

Air  Mission  Commander  to  Battle 
Captain  and  Ground  Commander 

5.1.1 

Update  Situation  as  needed 

Between  Aircrew  and  TOC  and  Aircrew 
and  Ground  Commander 

5.2 

Visually  ID  location  of  Friendlies 

Aircrew  to  Ground  Commander 

5.2.1 

Verbally  Confirm  location 

Aircrew  to  Ground  Commander 

5.2.2 

Visual/Digital  (Blue  Force  Tracker)  if  equipped  or 
possible 

Aircrew  to  Ground  Commander 

(continued) 
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Mission  Event 

PI  Number 

PI  Title 

Position 

5.3 

Develop  Plan/Scheme  of  Maneuver 

Aircrew  to  Ground  Commander 

5.3.1 

Establish  Task  and  Purpose  for  Aviation  Assets 

Ground  Commander 

5.3.2 

Establish  clearance  of  fires  authority 

Between  Aircrew  and  Ground 
Commander 

5.3.3 

Coordinate  Designation  and  Shooter  Duties 

Between  Aircrew  and  appropriate  assets 

5.3.4 

Discuss  plan  among  aircrews 

Aircrew 

5.3.5 

Recommend  course  of  action  to  Ground 
Commander 

Aircrew  and  Ground  Commander 

5.4 

Provide  security  in  accordance  to  unit  SOP  (this 
can  be  a  part  of  all  the  sections) 

Wingman 

5.5 

Develop  the  situation 

Aircrew 

5.5.1 

Use  UAS  data  to  develop  the  situation 

Aircrew  and  Ground  Commander 

5.5.2 

Establish  UAS  command  Relationship  (Direct 
Support,  OPCON,  etc) 

Aircrew  and  Ground  Commander 

6  Target  Acquisition 
(In  parallel  with  on 
station  tasks) 

5.6 

Communicate  Differences  in  Pattern  of  Life  as 
appropriate 

Aircrew 

6.1 

Communicate  Last  Known  Position  and 
Description  of  Target 

Ground  Commander 

6.1.1 

Request  this  information  if  not  given  freely 

Air  Mission  Commander 

6.2 

Begin  Search  for  Target 

Flight  Team 

6.2.1 

Incorporate  the  ISR  Plan 

Flight  Team 

6.2.2 

Visual 

Flight  Team 

6.2.3 

Sensor 

Flight  Team 

6.2.3. 1 

Choose  proper  sensor  given  ambient  conditions 

Flight  Team 
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Mission  Event 

PI  Number 

PI  Title 

Position 

6.2. 3.2 

Share  sensor  feeds  if  required 

Flight  Team 

6.2.4 

Recognize  threats 

Flight  Team 

6.2.4. 1 

Utilize  Appropriate  Standoff  Distance 

Flight  Team 

6.3 

Announce  target  in  sight 

Flight  Team 

6.3.1 

Wingman  confirm  target 

Flight  Team 

6.4 

Communicate  Target  Acquisition  to  ground  forces 

Aircrew  and  Ground  Commander 

6.5 

Confirm  target  with  appropriate  marking  technique 
for  Ground  Commander  using  SOP 

Aircrew  and  Ground  Commander 

7  Apply  ROE 

7.1 

Confirm  Ground  Commanders  intent 

Ground  and  Air  Commander 

7.1.1 

If  apply  lethal,  determine  hostile  intent 

Ground  and  Air  Commander 

7. 1.1.1 

Ground  Commander  or  AMC  must  confirm  hostile 
intent 

Ground  and  Air  Commander 

7.2 

Discuss  lethal  nonlethal  COAs 

Ground  and  Air  Commander 

7.3 

Discuss  proportionality 

Ground  and  Air  Commander 

7.3.1 

Desired  effect  accomplished  with  minimal 
collateral  damage 

Ground  and  Air  Commander 

7.3.2 

Weapon  choice  made  and  fires  coordinated 

Ground  and  Air  Commander 

7.3.3 

Engagement  Scheme  of  Maneuver 

Ground  and  Air  Commander 

7.4 

Discuss  collateral  damage 

Ground  and  Air  Commander 

7.4.1 

Minimum  safe  distance  for  weapon  effect 

Ground  and  Air  Commander 

7.5 

Make  Shoot/Don’t  Shoot  Decision 

Appropriate  Command  Element 

7.5.1 

Communicate  Decision  to  ground 

Air  Mission  Commander 

7.5. 1.1 

If  Don’t  Shoot,  continue  to  observe 

Flight  Team 

8  Clearance  of  Fires 

8.1 

Request  Clearance  of  Fires  from  Ground 
Commander 

Air  Mission  Commander 

(continued) 
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Mission  Event 

PI  Number 

PI  Title 

Position 

8.1.1 

Clearance  Received  and  Acknowledged  by 
appropriate  authority 

Ground  Commander 

8. 1.1.1 

Cleared  Hot 

Ground  Commander 

9  Employ  Weapon 
System 

8.2 

Verbally  Communicate  Weapons  release  clearance 
within  flight 

Air  Mission  Commander 

9.1 

Fire  weapon  based  on  SOP  and  previous  plan 

Aircrew 

9.1.1 

Flight  lead  sets  inbound  and  formation 

Aircrew 

9.1.2 

Wingman  provides  overwatch  and  cover 

Aircrew 

9.1.3 

Applies  appropriate  weapons  engagement 
technique  based  on  SOP 

Aircrew 

9. 1.3.1 

Employ  appropriate  combined  anns  technique. 

Aircrew 

9.1.4 

Flight  lead  calls  engaging 

Aircrew 

9. 1.4.1 

Wingman  acknowledges 

Aircrew 

9.1.5 

Flight  lead  calls  break 

Aircrew 

9. 1.5.1 

Wingman  acknowledges 

Aircrew 

9.1.6 

Wingman  calls  engaging  (if  required) 

Aircrew 

9. 1.6.1 

Flight  lead  acknowledges 

Aircrew 

9.2 

Determine  Effects  of  Weapons  and  Objective  Met 

Aircrew 

9.2.1 

Communicate  to  Ground  Commander 

Air  Mission  Commander 

9.3 

Detennine  Health  State  of  Aircraft  (if  applicable) 

Flight  Team 

9.3.1 

If  damage,  Go/No-go 

Aircrew 

9.3.2 

If  no-go  choose  course  of  action 

Flight  Team 

9.3.2. 1 

Coordinate  with  ground  forces  and  higher 

Air  Mission  Commander,  Ground 
Commander 

(continued) 
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Mission  Event 


PI  Number  PI  Title 


Position 


1 0  Battle  Damage 
Assessment 


1 1  Obtain  Revised 
Task  and  Purpose 


End  of  Mission  Phase 


10.1 

Give  BDA  to  Ground  Commander  as  per  unit  SOP 

Air  Mission  Commander 

10.1.1 

Weapon  Effect 

Air  Mission  Commander 

10.1.2 

Collateral  Effects 

Air  Mission  Commander 

10.2 

Give  BDA  to  TOC  as  per  unit  SOP 

Aircrew  Member 

10.2.1 

Weapon  Effect 

Aircrew  Member 

10.2.2 

Collateral  Effects 

Aircrew  Member 

11.1 

Aircrew  Determine  FARM  (Fuel  Ammo  Rockets 
Missiles) 

Air  Mission  Commander  Directed 

11.1.1 

AMC  Reports  Status  to  higher  and  ground 

Air  Mission  Commander 

11.1.2 

Determine  Go/No-go  Status 

Flight  Team 

11.2 

Obtain  Next  Mission 

Flight  Team 

11.2.1 

If  current  mission  incomplete  coordinate  with 
Ground  Commander 

Air  Mission  Commander  to  Ground 
Commander 

11.2.2 

If  current  mission  complete  coordinate  with  TOC 

Air  Mission  Commander  to  Battle 
Captain 

11.3 

Egress  Per  Unit  SOP  and  APG  (Area  Procedures 
Guide) 

Air  Mission  Commander  to  Battle 
Captain 

11.3.1 

Address  METTTC 

Air  Mission  Commander  to  Battle 
Captain 

11.4 

Ingress  to  next  mission  if  appropriate 

Air  Mission  Commander  to  Battle 
Captain 

(continued) 
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Mission  Event 

PI  Number 

PI  Title 

Position 

Post  Flight  Tasks  and  AAR  Phase 

12  Post  Mission 

12.1 

Post  Flight  Mission  Tasks  per  SOP 

Flight  Team,  Battalion 

12.1.1 

Aircrew  conducts  post  flight  on  aircraft 

Flight  Team,  Battalion 

12.1.2 

Battalion  closes  AMR 

Flight  Team,  Battalion 

12.2 

Conduct  Debrief  in  accordance  with  unit  SOP 

Air  Mission  Commander  and  Battalion 

S2  Section  at  a  minimum 

12.2.1 

Notes 

Air  Mission  Commander  and  Battalion 

S2  Section  at  a  minimum 

12.2.2 

Video 

Air  Mission  Commander  and  Battalion 

S2  Section  at  a  minimum 

12.2.3 

Provide  INPUT  to  the  Story  Board 

Aircrew  to  Battalion  Staff 

12.2.4 

Create  Story  Board  (if  trigger  pulled) 

Battalion  Staff 

12.3 

Conduct  AAR  in  accordance  with  unit  SOP 

Aircrew 

12.3.1 

Notes 

Aircrew 

12.3.2 

Video 

Aircrew 

12.3.3 

Clear  Concise  and  Complete 

Aircrew 

12.3.4 

Participatory 

Aircrew 
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APPENDIX  D 

PROTOTYPE  OBSERVER-BASED  PERFORMANCE  MEASURES 
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Army  Aviation  Collective  Tasks  (AACT) 

Flight  Team  Performance  Measures 


1.1  Coordination  for  Brief  Preparation 

1.1.1  Coordinate  with  Adjacent,  Higher,  and  Lower  Units 

1.1.2  Adherence  to  SOP  Battle  Rhythm 


1 .  Does  the  S3  Section  brief  the  required  elements  of  the  Operation  Summary  (past,  current,  future) 
during  the  pre-mission  brief? 


1  2  3  4  5 

Staff  does  not  brief  all  Staff  briefs  all  required  Staff  briefs  all  required 

required  elements  elements,  but  not  all  elements  in  an  accurate 

accurate  or  timely  and  timely  manner 


□  N/A 

a  n/o 


a.  If  applicable,  which  required  elements  were  missed? 
L  Fires 


L  Airspace 

Signal  (Call  signs, 
grids,  frequency) 

L  Weather 

Air/Ground  scheme 
of  maneuver 

0  Timelines 


L  ISR  platforms 
L  Other  (specify): 


b.  Does  the  Battle  Captain  provide  updates  and  verify  required  information? 


1  2  3  4  5 

Battle  Captain  does  not  Battle  Captain  briefs  Battle  Captain  talks  to  all 

check  for  updates  information;  some  is  relevant  units  to  get 

current  but  not  all  updates;  verifies  plan 

prior  to  the  air  crew  brief 


□  N/A 

□  N/O 
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2.  Does  the  Battalion  staff  brief  the  required  Intelligence  elements  during  the  pre-mission  brief? 


□  N/A 

□  N/O 
5 

1 

i  ; 

Staff  does  not  brief  all 
required  elements 

C 

V 

Staff  briefs 
elements, 
accurate 

3 

all  required 
but  not  all 
or  timely 

L 

el 

;  \ 

\  < 
Staff  briefs  all  requirec 
ements  in  an  accurate 
and  timely  mannei 

a.  If  applicable,  which  required  elements  were  missed? 

□  Last  12-24hrs 

0  Last  24-74  hrs 

:lf;  Most  Likely/Most  Dangerous  Enemy  Course  of  Action 
0  Refine  and  Update  PIR  (BOLO) 

®  Terrain  Analysis 

0  Other  (specify): 


1.2  Operation  Summary  (Past,  Current,  Future) 

3.  Does  the  flight  incorporate  the  elements  of  the  Operation  Summary  in  their  pre-mission  planning? 


1  2 
Flight  does  not 
incorporate  all  information 
from  the  brief 


3 

Flight  incorporates  some 
information  from  the  brief; 
plan  is  not  fully  developed 


□  N/A 

□  N/O 


Flight  incorporates  all 
information  from  the  brief; 
plan  is  fully  developed 


a.  If  applicable,  which  required  elements  were  missed? 

0;  Fires 

L  Airspace 

L  Signal  (Call  signs,  grids,  frequency) 

0  Weather 

L  Air/Ground  scheme  of  maneuver 
<M  Timelines 

ISR  platforms 
L  Last  12-24  hrs 

L  Last  24-72  hrs 

Intel  analysis  of  Enemy  Course  of  Action  in  area  of  operation 
(e.g.  most  likely,  most  dangerous) 

;ft  Refine  and  Update  PIR  (BOLO) 

JJ  Terrain  Analysis 

L  Other  (specify): 
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1.  3  Flight  Team  and  Aircrew  mission  preparation 


4.  Is  the  flight  performing  the  appropriate  tasks  prior  to  take  off? 

□  N/A 

□  N/O 

1  2  3  4  5 

Flight  does  not  complete  Flight  completes  all  tasks  Flight  completes  all  tasks 

all  tasks  in  time  allotted  in  time  allotted  in  time  to  perform 

rehearsals;  prepares  for 
contingencies 

1.3.1  Flight  Team  Brief 

5.  Does  the  flight  brief  the  mission  in  accordance  with  unit  SOP? 

□  N/A 

□  N/O 

1  2  3  4  5 

Flight  does  not  brief  all  Flight  briefs  all  required  Flight  briefs  and 

required  information  information,  but  not  all  discusses  all  required 

accurate  or  timely  information  in  an  accurate 

and  timely  manner 

a.  If  applicable,  which  required  elements  were  missed? 

L  Fires 

L  Airspace 

L  Signal  (Call  signs,  grids,  frequency) 

L  Weather 

L  Air/Ground  scheme  of  maneuver 

L  Timelines 

fj  ISR  platforms 

L  Last  12-24  hrs 

L  Last  24-72  hrs 

Intel  analysis  of  Enemy  Course  of  Action  in  area  of  operation 
(e.g.  most  likely,  most  dangerous) 

jd  Refine  and  Update  PIR  (BOLO) 

JJ;  Terrain  Analysis 

>||  Other  (specify): 
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6.  Does  the  flight  discuss  and  designate  roles  for  the  mission? 

□  N/A 
L  N/O 

1  2  3  4  5 

AMC  does  not  assign  AMC  assigns  roles  per  AMC  assigns  and  flight 

roles  SOP  discusses  roles  per  SOP 

1.3.2  Establish  weapons  release  authority 

7.  Do  the  aircrews  establish  weapons  release  authority  within  the  flight? 

□  Yes  □  N/A 

□  No  □  N/O 

1.3.3  Aircrew  Brief  in  accordance  with  SOP  and  checklist 

8.  Does  the  aircrew  follow  the  aircrew  brief  checklist  in  accordance  with  SOP? 

□  Yes  □  N/A 

H  No  □  N/O 

a.  If  no,  what  was  missed? 

Q  Mission  Overview  and  Flight  Plan 

vjj  Crew  actions,  duties,  and  responsibilities 

L  Emergency  Actions  and  Downed  Aircraft  Procedures 

Sj  Downed  Aircraft  Procedures 

Analysis  of  the  Aircraft  (logbook,  maintenance,  PPC) 

£J  SPINS 

L  Fighter  Management  and  Risk  Mitigation 
.g  Other  (specify) 


2.1  Flight  Team  Receipt  of  WARNO  and  Parallel  Mission  Planning 


9.  Does  the  flight  develop  and/or  adjust  their  mission  plan  according  to  information  provided  in  the 
pre-mission  brief  and  WARNO? 

□  N/A 

□  N/O 

1  2  3  4  5 

Flight  does  not  Flight  incorporates  some  Flight  incorporates  all 

incorporate  all  information  information  from  the  brief;  information  from  the  brief; 

from  the  brief  plan  is  not  fully  developed  plan  is  fully  developed 
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a.  If  applicable,  which  required  elements  were  missed? 

L  Fires 

If  Airspace 

L  Signal  (Call  signs,  grids,  frequency) 

L  Weather 

:|f  Air/Ground  scheme  of  maneuver 
0  Timelines 

L  ISR  platforms 

L  Last  12-24  hrs 

L  Last  24-48  hrs 

Intel  analysis  of  Enemy  Course  of  Action  in  area  of  operation 
(e.g.  most  likely,  most  dangerous) 

L  Refine  and  Update  PIR  (BOLO) 

Q:  Terrain  Analysis 
L  Other  (specify): 

2.2  Mission  Coordination 
2.2.1  Airspace 

1 0.  Does  the  flight  coordinate  airspace  deconfliction  measures  with  all  entities  prior  to  launch  (e.g. 
ROZs,  ground  units,  ATC,  UAS,  TOC,  etc.)? 

L  N/A 
L  N/O 


2.2.2  Coalition  Forces 


Flight  does  not  coordinate 
with  other  entities  during 
airspace  deconfliction 


Flight  coordinates  with 
other  entities; 
recommends  adjustments 
to  maximize  their  flight 


Flight  coordinates  with 
other  entities: 
recommends  adjustments 
to  maximize  all  assets 


11.  Does  the  flight  ensure  proper  coordination  (airspace,  cross  boundary  request,  transition  request, 
other  information  required  when  operating  in  or  through  coalition  area  of  operation)  with  coalition 


forces? 

_ _  [1  N/A 

1  2 

3 

L  N/O 

4  5 

Flight  does  not  coordinate 

Flight  coordinates  with 

Flight  coordinates  with 

with  coalition  forces  or 

coalition  forces  to  get 

coalition  forces  to  get 

seek  updates 

updated  information 

updated  information  and 

adjusts  as  needed 
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2.2.3  Obtain  UAS  Feeds  (if  possible) 

12.  Does  the  flight  request  and  use  UAS  feeds,  if  available  and  needed? 

□  N/A 

□  N/O 

1  2  3  4  5 

Flight  is  unaware  that  Flight  requests  UAS  data  Flight  requests  and  uses 

UAS  feeds  are  available  and  has  UAS  feeds  (e.g.  coordinates, 

available  for  use  discusses,  controls,  etc.) 

UAS  to  develop  the 
situation 

2.2.4  Update  Friendly  Situation 

1 3.  Does  the  flight  ensure  it  is  aware  of  changes  to  the  friendly  situation? 

□  N/A 
L  N/O 

1  2  3  4  5 

Flight  does  not  receive  or  Flight  receives  update  on  Flight  receives  updated 

ask  for  update  friendly  situation  friendly  situation  and 

adapts  as  needed 

2.2.5  Verify  Communication  Frequencies 

14.  Is  the  flight  utilizing  communication  in  accordance  with  the  Signal  Operations  Instructions  (SOI) 
and  SOP? 

L  N/A 
L  N/O 

1  2  3  4  5 

Flight  does  not  utilize  Flight  communicates  via  Flight  utilizes 

communications  in  non-secure  means  when  communications  in 

accordance  with  SOI  and  secure  is  appropriate  accordance  with  SOI  and 

SOP  SOP 

a.  Does  the  flight  use  the  correct  communication  frequencies? 

□  Yes  □  N/A 

□  No  □  N/O 

2.2.6  Verify  Callsigns 

15.  Does  the  flight  ensure  accurate  call  signs  for  units? 
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2 


□  N/A 

□  N/0 


1 

Flight  does  not  obtain 
correct  call  signs  prior  to 
mission  launch;  requests 
from  TOC  during  mission 


3 

Flight  verifies  call  signs 
are  accurate;  requests 
from  TOC  before  mission 
launch  if  incorrect 


4  5 

Flight  establishes  correct 
call  signs  and  frequencies 
from  parallel  mission 
planning 


2.2.7  Verify  Grid  Locations 


1 6.  Does  the  flight  ensure  accurate  grid  location  for  friendlies? 


1  1  1 

1  2 

Flight  does  not  obtain 
accurate  grid  locations 

3 

Flight  obtains  grid 
location  information; 
enters  into  system 

1  | 

4  5 

Flight  performs 
crosscheck  of  grid 
location  information; 
verifies  with  supported 
commander 

□  N/A 

□  N/O 


2.2.8  Threat  Update 

17.  Does  the  flight  request  a  threat  update? 

□  Yes  □  N/A 

□  No  □  N/O 


18.  If  required,  does  the  flight  request  additional  information  based  on  content  of  threat  update? 


i  ; 

Flight  does  not  request 
additional  information 

l 

Flight  requ 
additional  i 

3 

ests  some 
nformation 

L 

F 

\  5 

light  requests  action  to 
obtain  all  information 
required  to  fill  SA 

□  N/A 
L  N/O 


2.2.9  React  to  threat  update/change  COA 

19.  Does  the  flight  react  appropriately  to  the  threat  update? 


1  1  1 

1  2 

Flight  executes  pre¬ 
planned  mission 
regardless  of  threat 
update 

3 

Flight  recognizes 
adjustment  required; 
creates  hasty  plan 

4  5 

Flight  adjusts  tactics  to  fit 
threat  update 

L  N/A 
L  N/O 
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3 


Task  Quick  Reaction  Force  (QRF) 


20.  Was  a  QRF  required? 


0  Yes 


□  N/A 

□  N/O 


3.1  Attend  Final  Mission  Brief  as  required 

3.1.1  Final  Adjustments  to  Plan 

21 .  Does  the  Flight  adjust  their  plan  according  to  differences  between  WARNO  and  Final  Mission 
Brief 


□  N/A 

□  N/O 


Flight  does  not  review 
current  plan  for  changes 


Flight  acknowledges 
changes 


Flight  acknowledges 
changes  and  discusses 
impact  on  their  plan 


a.  Is  the  flight  performing  the  appropriate  tasks  prior  to  take  off? 


1  2 
Flight  does  not  complete 
all  tasks  in  time  allotted 


Flight  completes  all  tasks 
in  time  allotted 


□  N/A 

□  N/O 


Flight  completes  all  tasks 
in  time  to  perform 
rehearsals;  prepares  for 
contingencies 


b.  If  time  is  limited,  is  the  flight  performing  the  appropriate  tasks  prior  to  QRF? 


1  2 
AMC  and  aircrew  are 
either  all  attending  final 
brief  or  all  preparing  the 
aircraft 


Pilots  split  duties  between 
final  air  brief  and  final 
aircraft  preparation 


□  N/A 

□  N/O 


4  5 

AMC  and  PCs  attend  final 
air  brief  while  rest  of 
aircrews  ready  the  aircraft 


3.1 .2  Report  changes  to  aircrew. 

22.  Does  the  AMC  request  mission  updates  from  TOC  prior  to  launch? 


□  N/A 

□  N/O 


AMC  does  not  request 
changes 


AMC  requests  update 


AMC  request  update; 
verifies  update  with 
wingman 
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3.2 


Request  SITREP  on  Net 


23.  Does  the  AMC  request  SITREP  from  all  appropriate  resources  prior  to  launch? 


_ _ _ _ _  □  N/A 

□  N/O 

1  2  3  4  5 

AMC  does  not  check  in  AMC  checks  in  with  some  AMC  checks  in  with  all 

with  resources  resources  (e.g.  UAS,  resources  (e.g.  UAS, 

ground,  BAE)  ground,  BAE) 

24.  Does  the  AMC  confirm  receipt  of  SITREP  information  to  the  flight? 


1  2 
AMC  does  not  verify 
receipt  of  SITREP 
information  by  wingman 


3 

AMC  queries  wingman  for 
confirmation 


_ _  L  N/A 

L  N/O 

4  5 

AMC  verifies  wingman 
receives;  confirms 
understanding 


25.  Does  the  AMC  adjust  the  COA  in  response  to  changes  to  the  situation? 


1  2 
AMC  does  not  adjust 
COA  to  address  changes 
in  the  situation 


3 

AMC  makes  ad  hoc 
adjustments  to  the  COA 
to  address  changes  in  the 
situation 


_ _  L  N/A 

L  N/O 

4  5 

AMC  applies  contingency 
COAs  to  address 
changes  in  the  situation 


3.3  Launch  Order 


26.  Was  the  final  launch  order  communicated? 
□  Yes  □  N/A 

St  No  □  N/O 


a.  Does  the  flight  launch  as  soon  as  they  are  authorized? 

□  Yes  □  N/A 

□  No  □  N/O 


b.  If  no,  was  the  delay  communicated  to  the  TOC? 

_ I _ I _ I _ I _  JJ  N/A 

1  a  N/o 

1  2  3  4  5 

No  communication  of  Communicates  delay  to  Immediately  alerts  the 

delay  to  the  TOC  TOC;  no  indicates  of  new  TOC  will  not  make  launch 

launch  time  time;  explains  reason  for 

delay;  recommends  new 
launch  time;  provides 
updates  as  necessary 
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4  Enroute 


4.1  Call  Off  to  Battalion  TOC 

27.  Does  the  aircrew  commander  successfully  call  off  to  Battalion  TOC? 
□  Yes  □  N/A 

a  No  □  N/O 


4.1 .1  Battalion  Staff  log  the  flight  off  and  acknowledge 

28.  Does  the  Battalion  Staff  log  the  flight  off  and  acknowledge? 


1  2 
Staff  does  not 
acknowledge  without 
repeated  prompts;  does 
not  verify  information 


3 

Staff  acknowledges  call 
off  but  does  not  verify 
information 


_ _  L  N/A 

L  N/O 

4  5 

Staff  acknowledges  call 
off;  verifies  information  is 
correct 


4.1.2  Report  to  higher  Flight  Team  Departure 


29.  Does  the  Battalion  Staff  communicate  aircrew  takeoff  time  to  Brigade  TOC? 


1  2 
Staff  does  not 
communicate  aircrew 
takeoff  time 


3 

Staff  communicates 
aircrew  takeoff  time  but  it 
is  not  timely;  some 
incorrect  information  is 
passed 


_ _  □  N/A 

□  N/O 

4  5 

Staff  immediately  passes 
aircrew  takeoff  time; 
information  is  accurate 


4.1.3  Report  to  ground  forces  that  Flight  Team  is  Enroute 


30.  Does  the  Battalion  Staff  communicate  aircrew  takeoff  time  to  Ground  Unit  TOC  as  required? 


1  1  1 

1  2 

Staff  does  not 
communicate  aircrew 
takeoff  time  or  ETA;  no 
communication  with 
ground  unit  TOC 

3 

Staff  communicates 
aircrew  takeoff  time  and 
ETA 

:  i  1 

4  5 

Staff  immediately  passes 
aircrew  takeoff  time  and 
ETA;  provides  additional 
situation  awareness 
information  (e.g.  call  sign, 
number  of  aircraft, 
weapons  available,  etc.) 

D-l  1 


4.2  Aircrew  to  deconflict  airspace  as  required  (air  traffic  services) 

31 .  Does  the  flight  deconflict  the  airspace? 


1  2 
Flight  does  not  use 
information  sources  to 
anticipate  events;  does  not 
make  radio  calls 


3 

Flight  uses  information 
sources  to  anticipate 
events  but  does  not  push 
information  to  rest  of 
team 


_ _  □  N/A 

□  N/O 

4  5 

Flight  uses  information 
sources  to  anticipate 
events  and  proactively 
pushes  information  to  rest 
of  team  ) 


4.3  Monitor  and  Acknowledge  Updates 

32.  Does  the  flight  monitor  air  to  air  radio  communication? 


□  N/A 
L  N/O 


Flight  does  not  monitor  or 
acknowledge  air  to  air 
communications 


Flight  monitors  and 
disseminates  air  to  air 
communications 


Flight  monitors  air  to  air 
communications; 
discusses  and  addresses 
impacts  to  mission 


33.  Does  the  flight  monitor  ground  channels? 


Flight  does  not  monitor 
updates 

l 

Flight  mor 
disseminati 

3 

litors  and 
es  updates 

L 

FI 

dis 

1  j 

\  £ 
ight  monitors  updates; 
cusses  and  addresses 
impacts  to  mission 

□  N/A 

□  N/O 


4.4  Coordinate  Team  Tactics  with  wingman 

34.  Does  the  flight  coordinate  team  tactics  internally? 


1  2 
No  collaboration;  AMC 
dictates  decision  on  COA. 


3 

Collaborative  discussion, 
but  not  timely; 
overanalyzes  decision. 


_ _  □  N/A 

L  N/O 

4  5 

Collaborative  discussion 
is  short  and  concise; 
considers  all  variables 
(e.g.,  loiter,  orbit, 
approaches,  threats); 
timely  decision  on  COA 
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4.4.1  Loiter  or  Holding  Area 

35.  Does  the  flight  select  holding  area? 


_ 1 _ 1 _ 1 _ 1 _ 

1  2 
Flight  selects  area 
without  considering 
tactical  implications  (e.g., 
too  small,  indefensible, 
cannot  communicate) 

3 

Flight  selects  an  area; 
area  is  suitable  but  not 
optimal 

;  i  j 

4  5 

Flight  selects  an  area; 
considers  size,  comms, 
defensibility,  etc.  to 
identify  optimal  location 

a.  If  applicable,  what  tactical  implications  were  missed? 
0:  Concealment 


L  Obstacles 
L  Key  terrain 

L  Approach  and  departure  directions 

jj  360°  Security 

L  Other  (specify): 

36.  Does  the  flight  select  loiter  area? 


_ 1 _ 1 _ 1 _ 1 _ 

1  2 
Flight  selects  area 
without  considering 
tactical  implications  (e.g., 
too  small,  indefensible, 
cannot  communicate) 

3 

Flight  selects  an  area; 
area  is  suitable  but  not 
optimal 

1  1  : 

4  5 

Flight  selects  an  area; 
considers  size,  comms, 
defensibility,  etc.  to 
identify  optimal  location 

a.  If  applicable,  what  tactical  implications  were  missed? 
□  Size 


0  Suitable  location 

U  Communication  availability 
0  Altitude  for  loiter 

0  Pattern  of  loiter 

L  Time  to  target 

□  Other  (specify): 
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4.4.2  Deconfliction  Measures 

37.  Does  the  flight  develop  the  appropriate  flight  deconfliction  measures? 
□  Yes  □  N/A 

C  No  □  N/O 


4.4.3  Delegation  of  coordination  and  flight  related  duties  (e.g.  Communications) 

38.  Does  the  flight  delegate  and  coordinate  flight  related  duties  (e.g.,  communication)  in  response  to 
a  change  in  the  mission? 


1  2 
Flight  does  not  have  a 
plan;  confusion  over 
assigned  tasks 


3 

Flight  continues  with 
assigned  flight  duties; 
adapts  to  current  situation 


_ _  L  N/A 

L  N/O 

4  5 

Flight  reassigns  duties  to 
suit  situation 


4.5  Adherence  to  the  SOP 


39.  Does  the  flight  adhere  to  the  SOP? 


1  2 
Flight  does  not  follow 
SOP;  does  not 
communicate  deviation 
from  SOP 


3 

Flight  follows  SOP  and 
adapts  within  its 
constraints 


4  5 

Flight  understands  SOP 
and  deviates  when 
necessary; 
communicates  deviation 
for  flight  internal  SA 


□  N/A 
B  N/O 


40.  If  a  deviation  was  required,  did  the  flight  appropriately  deviate  from  the  SOP? 


1  2 
Flight  did  not  recognize 
need  for  deviation;  did  not 
deviate  from  the  SOP 


Flight  recognized  need 
for  deviation;  did  not 
appropriately  deviate 
from  the  SOP 


4  5 

Flight  recognized  need 
for  deviation;  deviated 
appropriately  from  the 
SOP 


L  N/A 
H  N/O 


4.5.1  Formation 


41.  Does  the  flight  discuss  vertical  and  lateral  displacement,  tactics,  and  protection  of  flight? 


1  2 
No  detailed  plan  or 
discussion 


3 

Defines  all  measures  but 
does  not  discuss 
adjustments 


_ _  L  N/A 

L  N/O 

4  5 

Defines  all  measures  and 
discusses  required 
adjustments 
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42.  Does  the  flight  adhere  to  the  flight  formation  as  briefed? 


1  2 
Flight  maintains  loose 
formation;  no  discussion 
on  the  formation  or 
changes  to  the  formation 


3 

Flight  sets  formation 
based  on  briefed  tactics 


_ 1 _ u  □  N/A 

'  □  N/O 

4  5 

Flight  sets  formation 
based  on  briefed  tactics; 
constantly  updates 
formation  based  on 
current  situation 


4.5.2  Flight  Duties 

43.  Does  the  flight  adhere  to  the  flight  duties  required  for  the  mission? 


1  2 
Flight  does  not  adhere  to 
flight  duties 


3 

Flight  adheres  to  flight 
duties;  no  further 
coordination  or  backup 


4  5 

Flight  adheres  to  flight 
duties;  adjusts  to  mitigate 
task  overload  and  based 
on  current  situation 


L  N/A 
L  N/O 


4.5.3  Communication  Protocol 


44.  Does  the  flight  follow  appropriate  communication  protocol? 


1  2 
Flight  uses  some 
available  communication 
tools  (e.g.,  data,  voice); 
does  not  use  pro-words 
or  brevity  codes  as 
required;  stepovers  occur 


3 

Flight  uses  some 
available  communication 
tools  (e.g.,  data,  voice); 
uses  pro-words  and 
brevity  codes 


_ _  L  N/A 

1  L  N/O 

4  5 

Flight  uses  appropriate 
communication  tools 
(e.g.,  data,  voice);  uses 
pro-words  and  brevity 
codes;  messages  contain 
appropriate  level  of  detail 


4.5.4  SOP  Driven  Communication  (FARM  etc.) 

45.  Does  the  flight  use  the  SOP  to  dictate  the  right  communication  flow? 


1  2 
Flight  does  not  follow  the 
proper  SOP  report  order 


3 

Flight  follows  the  proper 
SOP  report  order 


_ _  □  N/A 

□  N/O 

4  5 

Flight  follows  the  proper 
SOP  report  order;  weighs 
the  information  against 
the  plan  and  uses  it  to 
adjust  the  plan 
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4.5.5 


Tactics 


46.  Does  the  flight  develop  appropriate  tactics  if  there  is  misalignment  between  SOP  and  situation? 


1  2 
Flight  does  not  recognize 
the  misalignment;  makes 
no  changes  to  SOP  or 
tactics 


J _ _ _  □  N/A 

'  □  N/O 

3  4  5 

Flight  recognizes  Flight  recognizes 

misalignment;  tries  to  fit  misalignment; 

the  SOP  to  the  situation  appropriately  modifies  the 

tactics  based  on  the 
environment 


4.6  Execute  Air  Ground  Integration  Checklist 

4.6.1  Check-in  with  Ground 

4.6. 1.1  Number  of  Aircraft 

4.6. 1.2  Number  and  Type  of  Weapons  System  Available 

4.6. 1.3  Station  Time 

4.6.1. 4  Request  SITREP 

47.  When  does  the  flight  make  the  check-in  call  to  Ground? 


1  2  3 

On  arrival  Before  they  arrive  in  the 

AO 


_ _  L  N/A 

L  N/O 

4  5 

Immediately  upon  radio 
communication  range 


48.  Does  the  flight  check  in  with  ground? 


L  N/A 
L  N/O 


Flight  does  not  know 
proper  format;  misses 
multiple  items 


Flight  checks  in  with  all 
appropriate  items;  check¬ 
in  not  done  in  correct 
order 


Flight  checks  in  with  all 
appropriate  items;  check¬ 
in  done  in  correct  order 


a.  Which  items  were  missed? 

Q  Call  sign 

□  T ype  and  N u m ber  of  Ai rcraft 

□  Type  and  Number  of  Weapons  System  Available 
if  Station  Time 

□  Request  SITREP 

b.  Did  the  ground  acknowledge  aircraft? 

□  Yes  □  N/A 

is  No  -M  N/O 
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c.  Did  the  ground  send  SITREP  to  flight? 
E5  Yes  □  N/A 

C  No  □  N/O 


4.6.2  Receive  SITREP  from  Ground 

4.6.2. 1  Frontline  Trace  and  unit  composition 

4.6.2.2  Markings 

4.6.2. 3  Updates  to  the  situation 

49.  Does  the  flight  receive  the  SITREP  from  Ground? 


1  2 
Flight  does  not  receive 
SITREP;  does  not  follow 
up  with  request 


1 _ ^ _ I _ _ I _ B  N/A 

L  N/O 

3  4  5 

Flight  receives  SITREP  Flight  receives  SITREP; 

verifies  completeness 
and  requests  information 
pertinent  to  mission 


4.6.2.4  Obtain  UAS  feed  (if/when  possible) 

4.6.2.4.1  Goal  is  to  obtain  as  soon  as  possible  in  mission  thread 


50.  If  UAS  feed  was  available,  when  does  the  flight  request  it? 


_ _ _ _ _  □  N/A 

□  N/O 

1  2  3  4  5 

Flight  does  not  request  Flight  requests,  but  not  as  Flight  requests  as  soon  as 

soon  as  possible  possible  in  mission  thread 


4. 6.2. 5  Immediate  Airspace  Deconfliction  and/or  Avoidance  Measures 

51.  Does  the  flight  verify  the  airspace  is  clear  or  free  from  obstacles  (e.g.  helicopters,  fixed  wing, 
UAS,  artillery)? 


L  N/A 
L  N/O 


Flight  does  not  verify 
airspace  deconfliction 


Flight  reviews  ROZ 
information 


Flight  reviews  ROZ 
information;  makes  final 
call  to  verify  prior  to 
entering  airspace 
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5  Arrive  on  Station  -  Update  Situational  Awareness 


5.1  Communicate  Arrival  on  Station 

5.1.1  Update  Situation  as  needed 

52.  Does  the  flight  communicate  on-station  arrival? 
□  Yes  □  N/A 

6.  No  B  N/O 


5.2  Visually  ID  location  of  Friendlies 

53.  Does  the  flight  communicate  location  of  the  friendlies  to  the  ground  TOC? 


1  2 
Flight  does  not  pass 
correct  information  to 


3 

Flight  passes  correct 
information  to  TOC 


TOC 


5.2.1  Verbally  Confirm  location 

54.  Does  the  flight  confirm  location  of  friendlies  verbally? 


1  2 
Flight  does  not  verbally 
confirm  location  of 
friendlies 


3 

Flight  verbally 
confirms  location  of 
friendlies 


_ _  □  N/A 

□  N/O 

4  5 

Flight  passes  correct 
information  to  TOC; 
communicates 
implications  of  friendly 
position  on  mission 
execution 


_ _  □  N/A 

'  □  N/O 

4  5 

Flight  verbally 
confirms  location  of 
friendlies;  confirms 
among  aircrews  as 
well 


5.2.2  Visual/Digital  (Blue  Force  T racker)  if  equipped  or  possible 

55.  Does  the  flight  identify  the  location  of  friendlies  using  all  sources  available? 


□  N/A 

□  N/O 


Flight  does  not  use 
available  resources 


Flight  uses  some  of  the  Flight  maximizes  use  of  all 
available  resources  available  resources 
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5.3  Develop  Plan/Scheme  of  Maneuver 

5.3.1  Establish  Task  and  Purpose  for  Aviation  Assets 

56.  Does  the  flight  work  with  the  ground  to  establish  task  and  purpose  for  their  mission? 


□  N/A 

□  N/O 


Flight  does  not 
communicate  with  ground 
to  establish  task  and 
purpose 


Flight  communicates 
with  ground  to 
establish  task  and 
purpose 


Flight  collaborates 
with  ground  to  meet 
commander’s  intent 


5.3.2  Establish  clearance  of  fires  authority. 

57.  Does  the  flight  establish  who  has  clearance  of  fires  authority? 
□  Yes  H  N/A 

'M:  No  0  N/O 


5.3.3  Coordinate  Designation  and  Shooter  Duties 

58.  Do  the  aircrews  coordinate  and  designate  shooter  duties  within  the  flight? 


1  2 
Aircrews  do  not  have  an 
established  plan  for 
designating  shooter;  still 
discussing  roles. 


Aircrews  have  a  plan; 
assigns  duties 
according  to  plan 


_ _  □  N/A 

□  N/O 

4  5 

Aircrews  have  a  plan; 
continuously  updates 
plan  and  assigns  or 
reassigns  duties 
based  on  updates  to 
the  situation 


5.3.4  Discuss  plan  among  aircrews 

59.  Does  the  flight  discuss  applicable  changes  to  the  tactical  mission? 


□  N/A 

□  N/O 


Flight  does  not  discuss 


Flight  discusses; 
considers  changes  if 
required  but  tries  to  force 
initial  plan 


Flight  discusses;  makes 
changes  if  required 
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5.3.5  Recommend  course  of  action  to  Ground  Commander 


60.  Does  the  flight  recommend  course  of  action  to  Ground  Commander? 


1  2 
Flight  does  not 
recommend  course  of 
action;  moves  forward 
with  own  plan 


Flight  recommends 
course  of  action 
before  discussing 
with  ground 


□  N/A 

□  N/O 


Flight  establishes 
course  of  action 
based  on  discussion 
with  ground  over 
execution  of  task  and 
purpose 


5.4  Provide  security  in  accordance  to  unit  SOP  (this  can  be  a  part  of  all  the  sections) 

61 .  Does  the  flight  establish  security  within  the  flight? 


1  2 
Flight  does  not  establish 
or  follow  security  plan 


Flight  maintains  security 
using  assigned  duties 


□  N/A 

□  N/O 


Flight  maintains 
security  using 
assigned  duties; 

constant 
communication 
between  aircraft  to 
maintain  a  high  level 
of  security 


5.5  Develop  the  situation 

62.  Does  the  flight  continue  to  develop  the  situation  with  ground? 


Flight  continues  the 
mission  using  only  the 
information  provided  in 
check-in  with  ground  and 
their  recommended  COA 


Flight  continues  to 
observes  situation 
and  passes 
information  to  ground 


□  N/A 
0  N/O 


4  5 

Flight  continues  to 
observe  the  situation; 
constant  dialog  with 
ground  to  develop 
situation 


5.5.1  Use  of  UAS  data  to  develop  the  situation 

63.  If  dedicated  UAS  data  is  available,  does  the  flight  communicate  information  to  ground  forces? 


□  N/A 

□  N/O 


Flight  does  not  offer 
information  to  ground 


Flight  shares  some 
information  with  ground 
when  asked 


Flight  pushes  all  relevant 
and  available  information 
to  ground 
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64.  If  UAS  data  (beyond  dedicated  mission  support  UAS)  is  available,  does  the  flight  communicate 
information  to  ground  forces? 


_ _ _ _ _  □  N/A 

'  □  N/O 

1  2  3  4  5 

Flight  does  not  offer  Flight  shares  some  Flight  pushes  all  relevant 

information  to  ground  information  with  ground  and  available  information 

when  asked  to  ground 

65.  If  additional  ISR  data  is  available,  does  the  flight  share  information  with  ground  forces? 


_ _ _ _ _  □  N/A 

□  N/O 

1  2  3  4  5 

Flight  does  not  offer  Flight  shares  some  Flight  pushes  all  relevant 

information  to  ground  information  with  ground  and  available  information 

when  asked  to  ground 

5.5.2  Establish  UAS  command  relationship  (Direct  Support,  OPCON,  etc) 

66.  Does  the  flight  coordinate  with  UAS  control  authority? 


1  2 
Flight  does  not  establish 
relationship 


3 

Flight  establishes 
relationship 


4  5 

Flight  establishes 
relationship;  works  to 
develop  the  situation 


□  N/A 

□  N/O 


5.6  Communicate  Differences  in  Pattern  of  Life  as  appropriate 

67.  Does  the  flight  communicate  observed  differences  in  pattern  of  life? 


1  2 
Flight  doesn’t  recognize 
or  communicate 
differences 


3 

Flight  recognizes  and 
communicates 
differences 


4  5 

Flight  recognizes  and 
communicates  change; 
discusses  courses  of 
action  based  on 
differences 


□  N/A 

□  N/O 
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6.1  Communicate  Last  Known  Position  and  Description  of  Target 

6.1.1  Request  this  information  if  not  given  freely 


68.  Does  the  flight  communicate  with  ground  to  locate  the  target? 


1  2 
Flight  uses  last  SITREP 
and  begins  search;  does 
not  coordinate  further 
with  ground 


Flight  asks  ground  for 
target  location 


Flight  maintains 
dialogue  with  ground 
to  identify  targets 


□  N/A 

□  N/O 


6.2  Begin  Search  for  Target 

6.2.1  Incorporate  the  ISR  plan  (areas  of  observation) 

6.2.2  Visual 

6.2.3  Sensor 

69.  Does  the  flight  incorporate  the  ISR  plan? 


1  2 
Flight  does  not  use  all 
assets  available;  does  not 
distribute  areas  of 
observation 


Flight  coordinates  use  of 
sensors  and  available 
assets;  distributes  areas 
of  observation 


.0  N/A 
□  N/O 


4  5 

Flight  coordinates  use  of 
sensors  and  available 
assets;  distributes  areas 
of  observation;  factors  in 
METT-TC 


70.  Does  the  flight  actively  search  for  the  target? 


□  N/A 

□  N/O 


Flight  does  not  have  a 
plan  in  place  for  search; 
does  not  distribute  areas 
of  observation 


Flight  executes  an  ISR 
plan;  does  not  effectively 
distribute  areas  of 
observation 


Flight  executes  an  ISR 
plan  that  is  adapted  to 
search  environment 


6.2.3. 1  Choose  proper  sensor  given  ambient  conditions 

71.  Does  the  flight  use  the  appropriate  sensors  to  search  for  targets? 


Flight  does  not  use 
available  sensors 


Flight  uses  some  of  the 
available  sensors 


□  N/A 

□  N/O 


4  5 

Flight  maximizes  use  of  all 
available  sensors 
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6.2.3. 2  Share  sensor  feeds  if  required 

72.  Does  the  flight  share  sensor  feeds  among  aircrews? 


□  N/A 

□  N/O 


Flight  does  not  share 
sensor  feeds;  no  cross¬ 
talk  among  aircraft 


Flight  shares  sensor 
feeds;  no  cross-talk 
among  aircraft 


Flight  shares  sensor 
feeds;  cross-talk  focuses 
on  what  observations  are 
being  made 


6.2.4  Recognize  threats 

73.  Does  the  flight  effectively  recognize  threats? 


_ _ _ _ _  □  N/A 

□  N/O 

1  2  3  4  5 

Flight  does  not  recognize  Flight  recognizes  possible  Flight  recognize  threats; 

threats  threats  evaluates  potential  impact 

on  mission;  reports  as 
necessary 


6.2. 4.1  Utilize  Appropriate  Standoff  Distance 

74.  Does  the  flight  utilize  an  appropriate  standoff  distance? 


□  N/A 

□  N/O 


Flight  does  not  apply 
appropriate  standoff 
distance  to  search  based 
on  perceived  threat; 
gross  violations  of 
minimum  standoff 


Flight  considers 
appropriate  standoff 
distance;  small, 
unintended  violations  of 
minimum  standoff 


Flight  applies  appropriate 
standoff  distance;  no 
violations  occur 


6.3  Announce  target  in  sight 

75.  Does  the  aircrew  announce  that  the  target  is  in  sight? 


1  2 

3 

- 1 - 1 - 1 

4  5 

Aircrew  does  not 

Aircrew  announces 

Aircrew  announces 

communicate  detection  of 

detection 

detection;  pushes  target 

target  to  other  aircraft 

information 

□  N/A 

□  N/O 
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76.  Does  the  flight  use  correct  terms  to  announce  target  in  sight? 


□  N/A 

□  N/O 

1  2  3  4  5 

Flight  does  not  use  correct  Flight  uses  correct  terms  Flight  uses  correct  terms; 
terms;  not  concise  with  a  few  errors;  not  concise 

concise 

6.3.1  Wingman  confirm  target 

77.  Does  the  wingman  confirm  target  detection? 

□  N/A 

□  N/O 

1  2  3  4  5 

Wingman  does  not  Wingman  acknowledges  Wingman  establishes  PID 

acknowledge  detection  of  detection  of  the  target 

the  target 

78.  Does  the  wingman  use  correct  terms  to  confirm  target? 

□  N/A 

□  N/O 


6.4  Communicate  Target  Acquisition  to  ground  forces 

79.  Does  the  flight  confirm  target  acquisition  to  ground? 

□  N/A 

□  N/O 

1  2  3  4  5 

Flight  does  not  Flight  communicates  Flight  communicates 

communicate  target  target  acquisition  to  target  acquisition  and 

acquisition  to  ground  ground  while  maintaining  transfers  PID  to  ground 

PID 


1  2 
Wingman  does  not  use 
correct  terms;  not  concise 


Wingman  uses  correct 
terms  with  a  few  errors; 
not  concise 


Wingman  uses  correct 
terms;  concise 
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6.5  Confirm  target  with  appropriate  marking  technique  for  Ground  Commander  using  SOP 

80.  Does  the  flight  mark  the  target  to  confirm  its  location? 


_ _ _ _ _  □  N/A 

□  N/O 

1  2  3  4  5 

Flight  does  not  mark  Flight  marks  target  Flight  discusses  marking 

correct  target  or  uses  the  strategy  with  ground; 

incorrect  maker  marks  target 

appropriately 

7  Apply  ROE 


7.1  Confirm  Ground  Commander’s  intent 

81 .  Does  the  flight  consider  Ground  Commander’s  intent? 


□  N/A 

[I  N/O 
5 

1 

1 

t 

i  ; 

Flight  does  not  conside 
Ground  Commander’s 
intent 

l 

ir  F 

light  consic 
Command 

3 

lers  Grount 
ler’s  intent 

L 

i 

:  l 

\  i 

Flight  considers  anc 
confirms  Grounc 
Commander’s  intent 

7.1.1  If  apply  lethal,  determine  hostile  intent 

7. 1.1.1  Ground  Commander  or  AMC  must  confirm  hostile  intent 

82.  Does  the  flight  confirm  hostile  intent  prior  to  applying  lethal  force? 
U  Yes  Q  N/A 

-it  No  □  N/O 


a.  If  no,  why  not? 


L  N/A 

□  N/O 
5 

t 

i  ; 

Flight  does  not  discuss 
hostile  intent 

l 

1 

-light  assut 
intent;  relie 
rep< 

3 

Ties  hostile 
;s  on  other 
3rts 

L 

1 

1 

\  ; 

Flight  determines 
possible  hostile  intent 
talks  themselves  into  ii 
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7.2 


Discuss  lethal  nonlethal  COAs 


83.  Does  the  flight  discuss  lethal  and  nonlethal  COAs  with  Ground  Commander? 


1  2  3 

Flight  does  not  know  Flight  applies 

commander’s  intent;  acts  commander’s  intent 

without  considering 
commander’s  intent 


4  5 

Flight  applies 
commander’s  intent; 
communicates  alternative 
COAs  based  on  aerial 
perspective 


□  N/A 

□  N/O 


7.3  Discuss  proportionality 

7.3.1  Desired  effect  accomplished  with  minimal  collateral  damage 

84.  Does  the  flight  discuss  proportionality? 


_ _ _ _ _  □  N/A 

'  □  N/O 

1  2  3  4  5 

Flight  does  not  discuss  Flight  discusses  Flight  uses  proportionality 

proportionality  proportionality  considerations  to  select 

the  most  appropriate 
weapon 


7.3.2  Weapon  choice  made  and  fires  coordinated 

85.  Does  the  flight  choose  the  right  weapon  and  coordinate  fires? 


1  2 
Flight  misappropriates 
weapons;  does  not 
coordinate  fires  within  the 
flight 


3 

Flight  selects  appropriate 
weapons  system; 
coordinate  fires  within  the 
flight 


_ _  □  N/A 

□  N/O 

4  5 

Flight  selects  appropriate 
weapons  system; 
coordinates  fires  within 
flight  and  with  ground 


7.3.3  Engagement  Scheme  of  Maneuver 

86.  Does  the  flight  coordinate  an  engagement  scheme  of  maneuver? 


1  1  1 

1  2 

Flight  does  not  coordinate 
a  scheme  of  maneuver 

3 

Flight  coordinates  a 
scheme  of  maneuver 
applicable  for  the 
situation 

:  i  | 

4  5 

Flight  coordinates  a 
scheme  of  maneuver 
optimized  for  the  situation 

□  N/A 

□  N/O 


7.4  Discuss  collateral  damage 

7.4.1  Minimum  safe  distance  for  weapon  effect 
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87.  Does  the  flight  consider  collateral  damage? 


1  2 
Flight  does  not  consider 
collateral  damage 


I _ _ _  □  N/A 

□  N/O 

3  4  5 

Flight  discusses  collateral  Flight  uses  collateral 

damage  damage  considerations  to 

select  method  of 
engagement  and  weapon 
system 


7.5  Make  Shoot/Don't  Shoot  Decision 


88.  Does  the  flight  make  an  appropriate  shoot/don’t  shoot  decision  (e.g.  considers  commander’s 
intent;  hostile  intent;  collateral  damage)? 


1  2 
Flight  does  not  consider 
critical  variables  before 
making  shoot/don’t  shoot 
decision 


Flight  considers  critical 
variables  before  making 
shoot/don’t  shoot 
decision 


7.5.1  Communicate  Decision  to  ground 


89.  Does  the  flight  communicate  shoot/don’t  shoot  decision  to  ground? 


□  N/A 

□  N/O 


4  5 

Flight  considers  critical 
variables  before  making 
shoot/don’t  shoot 
decision;  develops 
alterative  COAs  if  don’t 
shoot  is  determined 


1  2 
Flight  does  not 
communicate  decision  to 
ground 


Flight  communicates 
decision  to  ground;  does 
not  provide  alternative 
COAs 


N/A 

N/O 


Flight  communicates 
decision  to  ground; 
provides  alternative 
COAs 


7.5. 1.1  If  Don’t  Shoot,  continue  to  observe. 

90.  Does  the  flight  continue  to  observe  the  target  after  don’t  shoot  decision  is  made? 


1  2 
Flight  goes  off  station 
after  don’t  shoot  decision 
is  made 


3 

Flight  remains  on  station 
and  continues  to  observe 
target 


_ _  □  N/A 

□  N/O 

4  5 

Flight  remains  on  station 
and  continues  to  develop 
the  situation 
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8  Clearance  of  Fires 


8.1  Request  Clearance  of  Fires  from  Ground  Commander 

91 .  Does  the  flight  request  clearance  of  fires  from  Ground  Commander? 


_ 1 _ _ 1 _ _ 1 _ _ 1 _ 

1  2 
Flight  does  not  request 
clearance  of  fires 

3 

Flight  considers  ROE; 
establishes 

friendly/enemy  positions; 
requests  clearance  of 
fires;  not  ready  to  effect 
the  target  while  going 
through  this  process 

;  1 

4  5 

Flight  considers  ROE; 

establishes 
friendly/enemy  positions; 
requests  clearance  of 
fires;  anticipates 
clearance  and  sets  up 
shot  during  this  process 

8.1.1  Clearance  Received  and  Acknowledged  by  appropriate  authority 

8.1. 1.1  Cleared  Hot 


□  N/A 
Hi  N/O 


92.  Does  the  flight  receive  acknowledgement  of  clearance  of  fires  from  ground  prior  to  engagement? 
D  Yes  □  N/A 

C  No  □  N/O 


8.2  Verbally  Communicate  Weapons  release  clearance  within  flight 

93.  Does  the  AMC  communicate  weapons  release  clearance  within  the  flight? 


1  2 
AMC  does  not 
communicate  weapons 
release  clearance  to  the 
rest  of  the  flight 


3 

AMC  communicates 
cleared  hot,  but  does  not 
confirm 

acknowledgement  from 
rest  of  the  flight 


_ _  □  N/A 

□  N/O 

4  5 

AMC  communicates  and 
confirms  cleared  hot  from 
rest  of  the  flight 
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9  Employ  Weapon  System 

9.1  Fire  weapon  based  on  SOP  and  previous  plan. 

9.1.1  Flight  lead  sets  inbound  and  formation 

94.  Does  the  flight  establish  inbound  heading  and  formation  in  accordance  with  briefed  tactics? 


1  2 
Flight  does  not 
communicate,  does  not 
follow  briefed  inbound 
heading  and  formation 


Flight  communicates 
follows  briefed  inbound 
heading  and  formation 


□  N/A 

□  N/O 


Flight  communicates  and 
follows  briefed  inbound 
heading  and  formation; 
trail  effectively  covers 
lead 


9.1.2  Wingman  provides  overwatch  and  cover 

95.  Does  the  wingman  provide  overwatch  and  cover? 


□  N/A 

□  N/O 


Wingman  does  not 
provide  overwatch  and 
cover;  fixates  on  target 


Wingman  provides 
overwatch  and  cover; 
uses  appropriate 
resources  to  aid  in 
overwatch 


Wingman  provides 
overwatch  and  cover; 
uses  appropriate 
resources  to  aid  in 
overwatch; 
communicates 
developing  situation  with 
lead  aircraft 


9.1 .3  Applies  appropriate  weapons  engagement  technique  based  on  SOP 

9.1 .3.1  Employ  appropriate  combined  arms  technique. 

96.  Does  the  flight  apply  appropriate  weapons  engagement  technique  based  on  threat  environment 
(METT-TC)? 


1  2 
Flight  does  not  apply 
appropriate  technique; 
target  not  prosecuted 


Flight  does  not  apply 
appropriate  technique; 
target  still  prosecuted 


□  N/A 

□  N/O 


Flight  applies  appropriate 
technique;  target 
prosecuted 


9.1.4  Flight  lead  calls  engaging 

9.1 .4.1  Wingman  acknowledges 

9.1.5  Flight  lead  calls  break 

9.1 .5.1  Wingman  acknowledges 

9.1.6  Wingman  calls  engaging  (if  required) 

9. 1.6.1  Flight  lead  acknowledges 
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97.  Does  the  flight  communicate  appropriately  throughout  the  engagement  ? 


_ _  □  N/A 

□  N/O 

4  5 

Aircrews  employ  clear, 
concise,  and  timely  radio 
calls;  acknowledgements 
made  in  a  timely  manner 

a.  Position  boxes 

E  Lead  Aircraft 

II  Trail  Aircraft 

I  Other  lead  (specify): 

I  Other  trail  (specify): 


1  2  3 

Aircrews  employ  Aircrews  employ  clear, 

inappropriate  radio  concise,  and  timely  radio 

chatter  calls 


9.2  Determine  Effects  of  Weapons  and  Objective  Met 


98.  Does  the  flight  determine  effects  of  weapons  and  meeting  of  engagement  objectives? 


1  2 
Flight  does  not  determine 
weapons  effects;  does 
not  communicate  effects 


3 

Flight  determines  effects 
of  weapons; 
communicates  within 
flight 


_ _  I  N/A 

□  N/O 

4  5 

Flight  determines  effects 
of  weapons; 
communicates  within 
fliqht;  determines  next 
COA 


9.2.1  Communicate  to  Ground  Commander 


99.  Does  the  flight  communicate  weapons  effect  to  ground? 


1  2 
Flight  does  not 
communicate  weapons 
effect 


3 

Flight  communicates 
weapons  effect;  not  clear 
and  descriptive 


_ _  I  N/A 

I  N/O 

4  5 

Flight  communicates 
weapons  effect; 
clear,  concise,  and 
descriptive 
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9.3  Determine  Health  State  of  Aircraft  (if  applicable) 

9.3.1  If  damage,  Go/No-go 

9.3.2  If  no-go  choose  course  of  action 

9.3.2. 1  Coordinate  with  ground  forces  and  higher 

100.  Do  the  aircrews  determine  the  health  states  of  their  aircrafts? 


1  2 
Aircrews  do  not  consider 
health  status  of  aircraft 


3 

Aircrews  check  warning 
and  caution  lights; 
determine  go/no-go 


_ _  □  N/A 

□  N/O 

4  5 

Aircrews  check  warning 
and  caution  lights; 
perform  visual 
inspections  of  others’ 
aircrafts;  discuss 
observed  damage 
locations;  determine 
go/no-go 


a.  Which  aircrews  did  not  determine  health  status? 

Checked 

Present  health 


Lead  aircraft 


□  S 


Trail  aircraft 


L  L 


Other  (text  box) 


L  L 


Other  (text  box) 


L  L 


b.  If  no-go,  do  the  aircrews  choose  an  appropriate  course  of  action? 


1  2 
Aircrews  react  to  no-go 
decision  without 
consideration  for  threat, 
other  aircrews,  etc.;  do 
not  coordinate  with  others 


3 

Aircrews  determine 
proper  course  of  action  in 
response  to  no-go 
decision;  coordinate  with 
ground  and  higher 
aviation 


_ _  □  N/A 

□  N/O 

4  5 

Aircrews  follow 
predetermined 
contingency  plan; 
coordinate  with  ground 
and  higher  aviation 
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10  Battle  Damage  Assessment 


10.1  Give  BDA  to  Ground  Commander  as  per  unit  SOP 

10.1.1  Weapon  Effect 

10.1.2  Collateral  Effects 

10.2  Give  BDA  to  TOC  as  per  unit  SOP 
10.2.1  Weapon  Effect 

101.  Does  the  flight  conduct  a  battle  damage  assessment? 


1  2 
Flight  does  not  conduct 
BDA;  assumes  target  is 
destroyed  without 
verification 


3 

Flight  evaluates  target; 
reports  BDA  to  ground 
and  command  elements 


_ _  □  N/A 

1  □  N/O 

4  5 

Flight  evaluates  target; 
proactively  reports 
complete  BDA  in  proper 
protocol  pushes/reports 
BDA  to  ground  and 
appropriate  command 
elements 


a.  Which  required  elements  were  missed? 

0  Supported  unit  (ground) 

0  Air  Element  TOC 
0  Other  (specify): 

b.  What  BDA  items  were  missed? 

L  Sending  to  right  place 

L  Alpha,  Call  sign  of  observing  source 
L  Bravo,  location  of  target 
L  Charlie,  time  strike  started  and  ended 
L  Delta,  percentage  of  target  coverage 

L  Echo:  itemized  destruction 


11  Obtain  Revised  Task  and  Purpose 


11.1  Air  Crew  Determine  FARM  (Fuel  Ammo  Rockets  Missiles) 

102.  Do  the  aircrews  discuss  FARM  (Fuel,  Ammo,  Rockets,  Missiles)? 


_ 1 _ 1 _ 1 _ 1 _ 

1  2 

Aircrews  do  not 
determine  FARM; 
assume  appropriate 
stores 

3 

Aircrews  make  accurate 
assessment  of  FARM 

1  | 

4  5 

Aircrews  make  accurate 
assessment  of  FARM; 
discuss  capabilities 
based  on  FARM 
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11.1.1  AMC  Reports  Status  to  higher  and  ground 

11.1.2  Determine  Go/No-go  Status  for  revised  task  and  purpose 

103.  Does  the  flight  advise  ground  on  go/no-go  status  for  the  revised  task  and  purpose? 


1  1  1 

1  2 

Flight  makes  decision 
based  on  one  aircrafts’ 
status  and  not  as  a  flight 

3 

Flight  makes  decision 
based  on  status  of  all 
aircraft 

4  5 

Flight  makes  decision 
based  on  status  of  all 
aircraft;  Develops  FARP 
rotation  for  revised  task 
and  purpose 

□  N/A 

□  N/O 


11.2  Obtain  Next  Mission 

11.2.1  If  current  mission  incomplete  coordinate  with  Ground  Commander 


104.  Does  the  flight  coordinate  with  ground  to  complete  original  mission? 


1  2 
Flight  does  not  recognize 
original  mission  is 
incomplete;  does  not 
coordinate  with  ground 


3 

Flight  recognizes  original 
mission  is  incomplete; 

waits  for  task  and 
purpose  tasking  from 
ground 


1 1 .2.2  If  current  mission  complete  coordinate  with  TOC 


_ _  □  N/A 

□  N/O 

4  5 

Flight  recognizes  original 
mission  is  incomplete; 
proactively  coordinates 
with  ground  to  obtain  task 
and  purpose 


105.  Does  the  flight  coordinate  with  aviation  TOC  after  being  released  from  ground? 


1  2 
Flight  does  not  coordinate 
with  aviation  TOC 


3 

Flight  coordinates  with 
aviation  TOC  after 
departing  the  AO 


4  5 

Flight  coordinates  with 
aviation  TOC  prior  to 
leaving  the  AO; 
proactively  requests 
follow-on  mission  based 
on  FARM 


L  N/A 
L  N/O 
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11.3  Egress  Per  Unit  SOP  and  APG  (Area  Procedures  Guide) 

11.3.1  Address  METTTC 

11.4  Ingress  to  next  mission  if  appropriate 

1 06.  Does  the  flight  tactically  egress  from  the  AO? 


1  1  1 

1  2 

Flight  chooses  straightest 
line  to  next  mission  or 
home 

3 

Flight  plans  proper  route 
to  egress/next  mission 
based  on  METT-TC 

i  1  1 

4  5 

Flight  varies  egress  route 
to  avoid  predictive 
behaviors;  adjusts  plan 
based  on  METT-TC 

12  Post  Mission 


12.1  Post  Flight  Mission  Tasks  per  SOP 

107.  Does  the  flight  log  down  with  aviation  TOC? 
if;  Yes  □  N/A 

C  No  □  N/O 


1 08.  Does  the  flight  conduct  post  flight  mission  tasks  per  SOP? 


1  2 
Flight  does  not  conduct 
post  flight  mission  tasks 


3 

Flight  follows  tasks 
according  to  SOP  as  a 
group 


4  5 

Flight  divides  tasking 
among  team  members; 
follows  tasks  according  to 
SOP 


L  N/A 
L  N/O 


12.1.1  Aircrew  conducts  post  flight  on  aircraft 

109.  Do  the  aircrews  conduct  post  flight  maintenance  and  tie  down? 


1  2 
Aircrews  skip  multiple 
steps;  skips  maintenance 
and  tie  down  altogether 


3 

Aircrews  conduct  post 
flight  maintenance  and  tie 
down 


_ _  □  N/A 

□  N/O 

4  5 

Aircrews  conduct  a 
thorough  post  flight 
maintenance  and  tie 
down;  reports  to 
maintenance  personnel 
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12.1.2  Battalion  closes  AMR  (Air  Movement  Request) 


110.  Does  the  Battalion  close  the  AMR? 


L  N/A 
L  N/O 


Battalion  does  not  close 
out  the  AMR;  no  log  down 
occurs 


Battalion  communicates 
closure  to  brigade 


Battalion  communicates 
closure  to  brigade  quickly 
and  without  errors 


12.2  Conduct  Debrief  in  accordance  with  unit  SOP 

12.2.1  Notes 

12.2.2  Video 

111.  Does  the  flight  conduct  debrief  in  accordance  with  unit  SOP? 


1  2 
Flight  provides  a  limited 
debrief;  numerous  errors 
and  omissions 

3 

Flight  provides  review  of 
the  mission  and  minimally 
debriefs 

4  5 

Flight  provides  clear, 
concise,  and  complete 
review;  reports  additional 
observations  not  related 
to  the  mission 

12.2.3  Provide  INPUT  to  the  Story  Board 

112.  Does  the  flight  provide  input  to  the  storyboard? 
□  Yes  □  N/A 

St  No  D  N/O 


12.2.4  Create  Story  Board  (if  trigger  pulled) 

113.  Does  the  Battle  Captain  create  the  storyboard? 


1  2 
Battle  Captain  does  not 
create  storyboard  or 
creates  storyboard  with 
multiple  errors;  submits  it 
late 


3 

Battle  Captain  focuses  on 
timeliness  rather  than 
quality  and  detail;  does 
not  consult  all  sources  of 
information 


_ _  L  N/A 

L  N/O 

4  5 

Battle  Captain  creates  a 
thorough  storyboard 
based  on  information 
gathered  during  the 
mission;  consults  all 
sources  of  information 
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12.3  Conduct  AAR  in  accordance  with  unit  SOP 

12.3.1  Notes 

12.3.2  Video 

12.3.3  Clear  Concise  and  Complete 

1 14.  Does  the  flight  conduct  an  AAR? 


1  2 
Flight  does  not  conduct 
an  AAR 


I _ _ _  L  N/A 

L  N/O 

3  4  5 

Flight  conducts  a  quick  Flight  conducts  a 

AAR;  touches  on  key  thorough  review;  records 

points  lessons  learned 


12.3.4  Participatory 

115.  Do  all  crew  members  participate  in  the  AAR? 


□  N/A 

□  N/O 
5 

f 

) 

i  ; 

Aircrew  members  are 
absent  during  the  AAR 

l 

P 

All  flight  me 
iresent  duri 

3 

jmbers  are 
ng  the  AAF 

L 

) 

V 

\  ; 
All  flight  members  are 
present  and  actively 
participate  in  the  AAF 
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APPENDIX  E 

DETAILED  DEFINITIONS  OF  PROTOTYPE  SYSTEM-BASED 
PERFORMANCE  MEASURES 


E-l 


Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Planning 

Mission  Event 

1  Pre-Mission  Planning 

PI 

1.1  Coordination  for  Brief  Preparation 

Status 

Future 

Reason  for  Classification 

Communication  in  the  TOC  (a  meeting).  If  electronic 
messages  are  used  to  communicate  operation  summary  data, 
there  is  a  possibility  of  measuring  whether  it  happened  or 
not.  Can  not  assess  quality  of  the  communication  via  system. 

Performance  Measure 

Did  the  area  of  responsibility  communicate  the  data  for  the 
intelligence  update? 

(a)  Previous  24 

(b)  Most  likely/Most  dangerous  enemy  course  of  action 

(c)  Refine  and  update  PIR  (BOLO) 

(d)  Terrain  analysis 

Required  Systems  for  Data 

Email,  mIRC,  or  CPOF  messages 

Required  Sim  Data 

Not  Applicable 

Assessment 

Message  is  sent  (communicated). 

Unit  of  Measurement 

Not  Applicable 

Acceptable  Performance 
Ranges  or  Values 

Message  is  sent  (communicated). 

Frequency  of  Occurrence 

Not  Applicable 

Triggering  Event(s) 

Not  Applicable 

Additional  Notes 

Cannot  assess  quality  of  messages.  Measure  requires  that 
inputs  are  passed  electronically. 

E-2 


Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Planning 

Mission  Event 

1  Pre-Mission  Planning 

PI 

1 .2  Operation  Summary  (Past,  Current,  Future) 

Status 

Future 

Reason  for  Classification 

Communication  in  the  TOC  (a  meeting).  If  electronic 
messages  are  used  to  communicate  operation  summary  data, 
there  is  a  possibility  of  measuring  whether  it  happened  or 
not.  Can  not  assess  quality  of  the  communication  via  system. 

Performance  Measure 

Did  the  area  of  responsibility  communicate  the  data  about: 

(a)  Fires  and  measures  of  effects? 

(b)  Airspace? 

(c)  Signal? 

(d)  Weather? 

Required  Systems  for  Data 

Email,  mIRC,  or  CPOF  messages 

(a)  AFATDDS  as  source  of  info 

(b)  TAIS  as  source  of  info 

(c)  Compare  SOI  and  SOP  vs  AMPS  —  Can  check  to  see  if 
data  were  entered  correctly  —  does  it  match?  booking  for 
errors.  If  can't  exchange  verbally,  what  has  been  done 
otherwise?  Parts  can  be  sent  digitally. 

Required  Sim  Data 

Not  Applicable 

Assessment 

Message  is  sent  (communicated). 

Unit  of  Measurement 

Not  Applicable 

Acceptable  Performance 
Ranges  or  Values 

Not  Applicable 

Frequency  of  Occurrence 

Not  Applicable 

Triggering  Event(s) 

Not  Applicable 

Additional  Notes 

Cannot  assess  quality  of  messages.  Measure  requires  that 
inputs  are  passed  electronically. 

E-3 


Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Planning 

Mission  Event 

1  Pre-Mission  Planning 

PI 

1.3  Flight  Team  and  Aircrew  Mission  Preparation 

Status 

Future 

Reason  for  Classification 

Did  the  flight  complete  all  appropriate  flight  preparation  data 
on  electronic  kneeboard? 

Performance  Measure 

Not  Applicable 

Required  Systems  for  Data 

Electronic  kneeboard  data. 

Required  Sim  Data 

All  required  values  filled  in. 

Assessment 

Not  Applicable 

Unit  of  Measurement 

Not  Applicable 

Acceptable  Performance 
Ranges  or  Values 

Before  take-off 

Frequency  of  Occurrence 

Not  Applicable 

Triggering  Event(s) 

Assessing  quality  of  knee  board  data  would  require  a 
standard  value  for  each  field. 

Additional  Notes 

Not  Applicable 

E-4 


Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Planning 

Mission  Event 

1  Pre-Mission  Planning 

PI 

1.3  Flight  Team  and  Aircrew  Mission  Preparation 

1 .3 .3  Aircrew  Briefs 

Status 

Potential 

Reason  for  Classification 

If  there  are  settings  in  the  aircraft  that  are  captured  in  the 
simulator,  can  assess  whether  or  not  the  aircrews  are  setting 
up  aircraft  here  like  they  are  instructed. 

Performance  Measure 

Did  they  initialize  X  system(s)? 

a.  Did  each  position  in  the  air  crew  set  up  required  radio 
frequencies? 

Required  Systems  for  Data 

DIS  Network 

Airspace  Weapons  Electronics  (AWE)  File  from  AMPS  in 
AH-64D 

Cockpit  HUD  Interactions 

Required  Sim  Data 

Transmitter  PDU  (Radio  ID,  Radio  Entity  Type,  Transmit 
State,  Input  Source,  Frequency,  other  fields  may  be 
required);  The  Transmitter  PDU  provides  detailed 
information  about  a  radio  transmitter. 

Assessment 

Not  Applicable 

Unit  of  Measurement 

Not  Applicable 

Acceptable  Performance 
Ranges  or  Values 

Not  Applicable 

Frequency  of  Occurrence 

Once  at  start  of  mission. 

Triggering  Event(s) 

Start  of  mission. 

Additional  Notes 

The  critical  question  is  what  pages  were  opened  by  the  crew 
once  they  loaded  their  AWE/ AMPS  files  in  the  cockpit.  The 
pages  should  be  tagged  by  seat  (not  sent  over  DIS  but  on  a 
separate  subsystem). 

E-5 


Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Planning 

Mission  Event 

3  Task  Quick  Reaction  Force  (QRF) 

PI 

3.2  Request  SITREP  on  Net  (prior  to  launch) 

Status 

Future 

Reason  for  Classification 

Communication  in  TOC.  May  be  via  radio  or  mIRC  (or  other 
C2  system).  If  electronic  messages,  might  be  able  to  do 
temporal  measures. 

Performance  Measure 

a.  Does  the  TOC  request  SITREP? 

b.  Does  the  TOC  request  a  SITREP  in  X  amount  of  time? 

Required  Systems  for  Data 

Email,  mIRC,  or  CPOF  messages  (SITREP);  analysis  on 
content  of  SITREP 

Required  Sim  Data 

Not  Applicable 

Assessment 

a.  Yes/No 

b.  X  amount  of  time. 

Unit  of  Measurement 

a.  boolean 

b.  time  (in  hours?) 

Acceptable  Performance 
Ranges  or  Values 

b.  Between  X  and  X  amount  of  time. 

Frequency  of  Occurrence 

Once  at  start  of  mission. 

Triggering  Event(s) 

Start  of  mission. 

Additional  Notes 

What  is  the  triggering  event?  How  do  we  know  a  SITREP  is 
requested? 

E-6 


Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Planning 

Mission  Event 

3  Task  Quick  Reaction  Force  (QRF) 

PI 

3.3  Launch  Order 

Status 

Potential 

Reason  for  Classification 

We  can  measure  when  the  aircrafts  take  off.  This  system 
measure  could  also  be  used  in  conjunction  with  an  observer- 
measure  that  looks  at  launch  order  (most  likely  situation). 

The  flight  needs  to  take  off  on-time  to  arrive  on-station  at  the 
appropriate  time. 

Performance  Measure 

Does  the  flight  launch  in  time  to  arrive  on-station  in  the 
agreed  upon  time  on  target? 

At  what  time  does  the  flight  launch? 

Required  Systems  for  Data 

DIS  Network;  Estimated  Time  On  Target  (TOT) 

Required  Sim  Data 

Entity  PDU  (Entity  Identification,  Force  Identification, 

Entity  Type,  Entity  World  Location).  Entity  PDU  (Entity 
Linear  Velocity).  The  Entity  Linear  Velocity  is  represented 
by  a  vector.  The  Linear  Velocity  vector  is  composed  of  three 
components  in  either  world  coordinate  or  entity  coordinate 
system  depending  on  the  field  in  the  Dead  Reckoning 
Algorithm  field.  The  three  components  represent  the  x,  y  and 
z  axis  of  the  entity.  Each  vector  component  is  in  meters  per 
second  squared. 

Assessment 

Compound  Measure:  Compare  reported  launch  order 
(observer-based)  with  time  of  take-off  (system-based)  and 
did  they  arrive  on-station  at  agreed  upon  TOT? 

Unit  of  Measurement 

Time  in  seconds;  meters  per  second  squared. 

Acceptable  Performance 
Ranges  or  Values 

Arrive  on-station  at  agreed  upon  TOT.  Good  performance  + 
or  -  3  minutes.  4-6  minutes  is  moderate  performance.  10 
minutes  late  is  a  fail. 

Frequency  of  Occurrence 

Once 

Triggering  Event(s) 

Not  Applicable 

Additional  Notes 

The  critical  issue  is  whether  you  take  off  with  enough  time  to 
be  on  station  when  you  need  to  be  there-this  is  really  a  time 
on  target  measure,  "here  is  your  mission-be  on  station  by 

X. . ."  Battle  Captain  gives  time,  then  crew  needs  to 
backward  plan.  It  is  the  responsibility  of  the  AMC  to  make 
sure  you  take  off  in  time. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Enroute 

Mission  Event 

4  Enroute 

PI 

4.1  Call  Off  to  Battalion  TOC 

Status 

Future 

Reason  for  Classification 

Communication  between  aircraft  and  TOC.  May  be  via  radio 
or  mIRC  (or  other  C2  system).  If  electronic  messages,  might 
be  able  to  do  temporal  measures  or  assess  if  they  are  at  a 
specific  phase  of  flight.  Might  be  a  trigger  measure  for  some 
other  system-based  measures. 

Performance  Measure 

Not  Applicable 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Not  Applicable 

Assessment 

Not  Applicable 

Unit  of  Measurement 

Not  Applicable 

Acceptable  Performance 
Ranges  or  Values 

Not  Applicable 

Frequency  of  Occurrence 

Not  Applicable 

Triggering  Event(s) 

Not  Applicable 

Additional  Notes 

Not  Applicable 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Enroute 

Mission  Event 

4  Enroute 

PI 

4.2  Aircrew  to  deconflict  airspace  as  required  (air  traffic 
services) 

Status 

Likely 

Reason  for  Classification 

We  can  check  to  ensure  they  do  not  break  restricted  air 
space.  We  would  need  to  know  beforehand  if  there  were  pre¬ 
determined  airspace  restrictions.  You  can  talk  your  way 
through  most  airspace.  Need  to  know  whether  or  not  broke 
the  airspace,  then  need  to  look  at  observer  notes  to  see 
whether  or  not  they  called  it. 

Performance  Measure 

a.  Does  the  aircrew  adhere  to  airspace  restrictions?  b.  Does 
the  aircrew  adhere  to  special  use  airspace? 

Required  Systems  for  Data 

TAIS  (for  airspace  restriction  data);  DIS  Network  (for 
aircraft  position  data) 

Required  Sim  Data 

Entity  Position  data;  Restricted  airspace  parameters  from 
TAIS.  Entity  PDU  (Entity  Identification,  Force 

Identification,  Entity  Type,  Entity  World  Location).  The 

Entity  Linear  Velocity  is  represented  by  a  vector  of  x,y,z 
axis  data. 

Assessment 

Compound  Measure:  If  airspace  restrictions  were  broken,  it 
will  be  important  to  assess  whether  or  not  a  radio  call  was 
made  (assess  via  observer-based  measures). 

Unit  of  Measurement 

Meters 

Acceptable  Performance 
Ranges  or  Values 

Acceptable  performance  is  not  breaking  restricted  airspace. 
Any  break  is  unacceptable. 

Frequency  of  Occurrence 

Constant 

Triggering  Event(s) 

Within  .1  mile  of  restricted  airspace  parameters 

Additional  Notes 

We  will  not  be  able  to  assess  this  fully  unless  the  observer 
records  that  a  radio  call  was  made  to  break  airspace.  Can  get 
predetermined  restrictions  from  TAIS.  AMPS  is  also  capable 
of  doing  it,  but  may  not  be  used.  Requests  for  airspace  all  go 
through  TAIS.  Cannot  enter  area  within  3K  of  ROZ  —  it  is 
represented  by  point  on  ground.  Can  be  varying  diameter 
often  plotted  as  a  circle,  but  can  be  other  shapes.  At  borders 
there  may  be  a  buffer.  In  ATX  there  are  sometime  cross 
boarder  situations  deliberately  put  in  scenario.  Note  that 
there  is  also  special  use  airspace  (in  addition  to  restricted). 
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Category  of  Data 
Mission  Phase 


Data  Required  for  System-Based  Measurement 
Enroute 


Mission  Event 
PI 

Status 

Reason  for  Classification 


Performance  Measure 

Required  Systems  for  Data 
Required  Sim  Data 


Assessment 

Unit  of  Measurement 

Acceptable  Performance 
Ranges  or  Values 

Frequency  of  Occurrence 

Triggering  Event(s) 


4  Enroute 

4.3  Monitor  and  Acknowledge  Updates 
Likely 

If  radio  calls  are  written  into  the  scenario,  will  have  a 
specific  number/frequency.  Then  we  need  to  see  if  aircraft 
returns  a  message  on  that  same  frequency,  and  how  long  it 
takes  to  do  that.  But,  the  way  in  which  a  transmission  ends  is 
important:  If  OVER,  a  response  is  required;  if  OUT,  no 
response  is  required.  Therefore,  this  measure  may  be  more 
appropriate  once  a  basic  level  of  voice  recognition  is 
available.  There  is  a  flight  internal  frequency  (designated 
frequency  for  that  mission  for  the  number  of  aircraft  in  that 
flight).  This  will  be  one  of  the  5  frequency  switches.  Length 
of  comm  should  not  be  a  factor,  and  may  be  tracking  5 
frequencies,  but  everyone  tracking  the  same  5  frequencies. 
SMEs  think  this  measure  is  more  likely  assessed  by 
observer. 

Did  the  flight  properly  divide  and  maintain  responsibility  for 
communication  channels? 

DIS  Network 

Knowledge  of  the  internal  flight  frequency  (Each  device  has 
a  seat  position  with  an  assigned  frequency);  Knowledge  of 
the  external  locations  frequency  (Ground  and  TOC);  Signal 
PDU  (Entity  ID,  Radio  ID,  Encoding  Scheme,  TDL  Type, 
Sample  Rate,  Data  Length) 

Each  seat  position  should  be  responding  to  a  proportionate 
number  of  transmissions.  For  example,  front  2  are 
responsible  for  Ground,  backseat  2  are  responsible  for  TOC, 
etc.) 

Counts;  Percentages 
Situation  dependent. 


Continuous,  computed  at  the  end  of  the  mission. 
Continuous. 


(continued) 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Additional  Notes 

Digital  messages  are  acknowledged  automatically.  On 
verbal,  should  respond  if  "over"  and  not  necessary  if  "out" 

(this  helps  with  preventing  stepping  on  each  other).  Crew 
brief  will  include  division  of  communication  responsibilities. 
They  should  divide  up  who  is  listening  to  what  (front  2, 
backseat  2,  and  wingman  2).  Point  is  to  divide  up  load. 
Measure  would  be  whether  divided  up  appropriately. 

Simulator  should  be  able  to  tell  you  who  has  what  channel 
set  up.  Some  may  also  be  delegated  to  electronic  tools,  chat, 
etc. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Enroute 

Mission  Event 

4  Enroute 

PI 

4.4  Coordinate  Team  Tactics  with  wingman 

Status 

Future 

Reason  for  Classification 

Cannot  get  at  quality  of  conversation  or  to  ensure  certain 
things  covered.  If  noticed  an  exchange  (not  just  one  person 
dictating)  may  be  indicative  of  good  performance  per  PI 
notes.  Talking  on  the  right  frequencies,  amount  of  time  spent 
in  conversation.  It  will  be  hard  to  isolate  team  tactics 
discussion  from  other  discussion.  A  lot  depends  on  how  well 
plan  before  take  off,  and  how  much  of  situation  has  changed. 

Performance  Measure 

Not  Applicable 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Not  Applicable 

Assessment 

Not  Applicable 

Unit  of  Measurement 

Not  Applicable 

Acceptable  Performance 
Ranges  or  Values 

Not  Applicable 

Frequency  of  Occurrence 

Not  Applicable 

Triggering  Event(s) 

Not  Applicable 

Additional  Notes 

This  is  low  priority  because  so  high  in  difficulty. 
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Category  of  Data 
Mission  Phase 


Data  Required  for  System-Based  Measurement 
Enroute 


Mission  Event 
PI 

Status 

Reason  for  Classification 


Performance  Measure 

Required  Systems  for  Data 
Required  Sim  Data 


Assessment 

Unit  of  Measurement 

Acceptable  Performance 
Ranges  or  Values 

Frequency  of  Occurrence 


4  Enroute 

4.4  Coordinate  Team  Tactics  with  wingman 
4.4.1  Loiter  or  Holding  Area 

Potential 

The  context  and  environment  of  the  mission  plays  a  large 
part  in  the  selection  of  the  loiter/holding  area  (and  whether  it 
happens  at  all).  Size  of  loiter  and  holding  area  is  measurable. 

Defensible?  Geometric  characteristics  we  can  calculate  (i.e. 
distance  from  enemy  SAMS  site,  enemy  forces  in  that  area, 
etc.).  If  loiter  area  has  certain  physical  characteristics  that 
can  be  measured,  we  can  possibly  develop  this  system-based 
measures.  We  may  not  be  able  to  tell  if  it  is  a  good  area 
based  on  things  like  ingress/egress  patterns  though...  Also, 
depending  on  mission  (e.g.  scout)  may  not  use  loiter/holding 
area  at  all. 

Is  the  loiter  or  holding  area  outside  of  weapons  engagement 
zones?  Are  they  using  a  terrain  feature? 

DIS  Network;  MSEL 

Location  in  relation  to  enemy;  Type  of  enemy's  weapon 
system.  Entity  PDU  for  Friendly  (Entity  Identification,  Force 
Identification,  Entity  Type,  Entity  World  Location,  Entity 
Capabilities,  Articulation  Parameters)  and  Entity  PDU  for 
Hostile  (Entity  Identification,  Force  Identification,  Entity 
Type,  Entity  World  Location,  Entity  Capabilities, 
Articulation  Parameters).  Location  in  relation  to 
building/mountain/etc. 

The  area  selected  should  be  as  close  as  possible,  without 
breaking  the  enemy  engagement  zone.  Having  a 
building/mountain  (something  that  will  obstruct  the  shot) 
between  you  and  the  enemy. 

Meters. 

Situation  dependent. 


Whenever  approaching  on-station  and/or  approaching  the 
enemy  (whenever  loitering  prior  to  mission  execution). 


(continued) 
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Category  of  Data  Data  Required  for  System-Based  Measurement 

Triggering  Event(s)  How  to  know  if  loitering:  Stay  in  one  area  for  more  than  a 

minute.  Size  of  loiter  area  varies  —  may  be  a  race  track 
pattern,  may  just  be  hovering. 

Additional  Notes  Will  be  context  dependent,  and  decided  by  AMC.  Issue  will 

be  whether  on  station  during  critical  time  (arrive  late,  leave 
early  due  to  fuel,  etc . . .).  If  larger  air  assault  mission  will 
have  preplanned  loiter  area  —  but  for  attack,  will  rarely 
preplan  loiter  area.  Air  assaults  are  set  up  ahead  of  time. 
Should  not  loiter  too  close  to  target  area  —  need  to  stand  off 
to  not  tip  off  enemy....  but  should  still  be  able  to  see  target 
from  long  way  off.  Should  loiter  within  sensor  range.  If  you 
know  where  aircraft  is,  and  where  FARP  is,  and  fuel,  can 
calculate  when  they  need  to  leave  —  this  may  or  may  not 
match  mission  requirements,  which  may  be  hard  to 
automatically  calculate...  fuel  check  should  be  done  shortly 
after  take  off...  (observer).  Breaking  off  will  be  done 
verbally...  Could  cue  observer  that  time  is  running  short  — 
but,  they  can  already  see  some  of  this.  700  pounds  (AH-60) 
is  lowest  they  should  go...  "bingo" 
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Category  of  Data 
Mission  Phase 


Data  Required  for  System-Based  Measurement 
Enroute 


Mission  Event 
PI 

Status 

Reason  for  Classification 

Performance  Measure 

Required  Systems  for  Data 
Required  Sim  Data 


Assessment 

Unit  of  Measurement 

Acceptable  Performance 
Ranges  or  Values 


Frequency  of  Occurrence 
Triggering  Event(s) 


4  Enroute 

4.5  Adherence  to  SOP 
4.5.1  Formation 

Potential 

If  an  average  distance  or  altitude  from  wingman  is  required, 
we  can  measure  that.  For  each  aircraft  the  Acceptable 
distance  is  different;  The  distance  for  a  wing  should  be  an 
appropriate  distance  to  see  and  still  effect  the  other  aircraft. 

a.  Does  the  flight  (lead  and  wing)  remain  in  line  of  sight? 

b.  How  much  time  does  a  wing  remain  out  of  line  of  sight? 

DIS  Network;  Mission  specifications  (aircraft,  objective) 

Position  of  lead  (Entity  PDU:  Entity  Identification,  Force 
Identification,  Entity  Type,  Entity  World  Location);  Position 
of  wing  (Entity  PDU:  Entity  Identification,  Force 
Identification,  Entity  Type,  Entity  World  Location); 
Acceptable  distance  to  see  and  still  effect  other  aircraft  (< 
quarter  mile);  Entity  PDU  (Entity  Linear  Velocity). 

Acceptable  distance  to  see  and  still  effect  other  aircraft  (< 
quarter  mile) 

Meters 

Good  data  would  be  average  distance  from  wingman. 
Combine  SOP  and  what  was  briefed  during  mission  to  assess 
performance.  But,  this  part  of  brief  can  be  changed  on  the 
fly.  Unit  SOP  will  determine  Acceptable  distances.  Brief 
different  formations  and  based  on  weather,  enemy,  etc.,  will 
determine  how  far  to  separate  aircraft.  Could  be  different 
every  flight,  and  for  phase  of  flight  too.  Try  not  to  write  too 
much  on  knee  board  card  esp.  when  in  theater.  When  flying 
for  awhile,  remember  those  numbers.  Talk  about  it  in  brief — 
may  spread  out  by  time  instead  of  3-5  rotar  wings  for 
Apache,  it  varies  greatly.  May  change  during  flight 
depending  on  situation. 

Continuous;  might  be  best  to  measure  any  deviations. 
Take-off 


(continued) 
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Category  of  Data  Data  Required  for  System-Based  Measurement 

Additional  Notes  Formation  flying  is  extremely  mission  dependent.  The 

following  data  is  rough  estimates  that  may  be  used  initially 
to  develop  a  measure.  But,  ultimately  parameters  must  be 
flexible.  A  set  formation  for  utility  aircraft  (Blackhawk, 
Chinook)  may  be  3  to  5  rotor  disks  apart  (this  is  a  rough 
estimate  they  use,  rotor  disk  is  diameter  of  the  helicopter 
blades);  attack  aircraft  does  not  use  this  standard.  A  common 
rale  of  thumb  is  to  stay  within  a  distance  where  you  can  see 
and  effect  the  other  guy.  Flowever,  there  is  no  set  standard. 
During  ingress,  should  be  whatever  is  required  to  support. 
But,  they  should  not  be  in  a  straight  line.  Also,  should  not  be 
a  kilometer....  should  be  close  enough  to  coordinate.  So,  lots 
of  latitude,  but  will  be  determined  by  AMC.  Could  provide 
OC  info  on  average  distance,  but  assessment  depends  on 
what  is  determined  by  AMC.  Could  also  report  altitude 
differences  —  criterion  is  whether  they  can  support  each 
other.  So,  best  may  be  just  to  give  stats  to  help  OCs...  Unit 
SOP  will  specify  distances  desired.  Note  that  this  is  about 
ingress.,  will  be  based  on  threat  analysis,  bottom  line  —  good 
to  provide  distances  and  altitudes,  but  assessment  TBD. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Enroute 

Mission  Event 

4  Enroute 

PI 

4.5  Adherence  to  the  SOP 

4.5.3  Communication  Protocol 

Status 

Potential 

Reason  for  Classification 

If  target  locations  and  geospatial  information  is  passed 
through  electronic  means,  we  may  be  able  to  capture.  Are  the 
right  targets  being  sent?  (target  locations,  geospatial 
information) 

Performance  Measure 

Does  the  flight  enter  in  the  updates  to  target  and  friendly 
locations?  Does  the  flight  enter  the  updates  to  ground  unit 
frequencies? 

Required  Systems  for  Data 

DIS  Network;  Blue  Force  Tracker  Device(s)  (FBCB2); 

FBCB2  to  DIS  gateway 

Required  Sim  Data 

Target  and  friendly  actual  positions  (from  scenario);  target 
and  friendly  coordinates  entered  in  the  cockpit  via  targeting 
system;  frequency  of  ground  unit; 

Assessment 

The  exact  position  of  the  friendlies  is  Acceptable;  anything 
else  is  unacceptable. 

Unit  of  Measurement 

Not  Applicable 

Acceptable  Performance 
Ranges  or  Values 

Accurate  data  entry  is  performed.  It  is  either  yes  or  no. 

Frequency  of  Occurrence 

On  entry  of  data  into  system. 

Triggering  Event(s) 

On  entry  of  data  into  system. 

Additional  Notes 

Pilots  fat  finger  in  the  position  in  the  cockpit;  BFT  link 
(targeting  system);  Blue  Force  Tracker  information  is 
generally  available  in  DIS  when  an  FBCB2  to  DIS  gateway 
is  available.  The  exact  form  of  the  DIS  packet  is  unknown;  if 
preplanned  mission,  might  put  target  in  AMPs.  More  likely 
on  kneeboard  packet  —  will  type  in  when  get  into  cockpit. 
When  close,  verify  with  sensors  that  image  and  visual  match. 

If  mismatch,  may  call  ground  to  verify  coordinate.  Note  that 
FCR  (Fire  Control  Radar)  can  give  a  lot  of  information  on 
targets  (e.g.,  Korea),  but  unlikely  to  be  used  much  in  current 
situation.  Will  use  TSD  page  to  get  plot  of  target  —  then  look 
at  sensors  —  verify  match.  Note  —  Army  uses  MGRS  to 
index  location  (may  or  may  not  use  lat/long). 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Enroute 

Mission  Event 

4  Enroute 

PI 

4.6  Execute  Air  Ground  Integration  Checklist 

Status 

Potential 

Reason  for  Classification 

Should  be  able  to  measure  whether  or  not  they  send 
messages  to  ground  (or  talk  to  them).  But  we  cannot  assess 
the  quality  of  that  communication  via  system-measures. 

Performance  Measure 

Not  Applicable 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Signal  PDU  (Entity  ID,  Radio  ID,  Encoding  Scheme,  TDL 
Type,  Sample  Rate,  Data  Length) 

Assessment 

Not  Applicable 

Unit  of  Measurement 

Not  Applicable 

Acceptable  Performance 
Ranges  or  Values 

Not  Applicable 

Frequency  of  Occurrence 

Not  Applicable 

Triggering  Event(s) 

Not  Applicable 

Additional  Notes 

Not  Applicable 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Enroute 

Mission  Event 

4  Enroute 

PI 

4.6  Execute  Air  Ground  Integration  Checklist 

4.6.1  Check-in  with  Ground 

Status 

Likely 

Reason  for  Classification 

Not  Applicable 

Performance  Measure 

Does  the  flight  initiate  the  air  ground  integration  checklist 
prior  to  arrival  on-station? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Prior  to  arrival  on-station  (as  soon  as  radio  contact);  Position 
of  on-station;  Entity  PDU  (Entity  Linear  Velocity);  Signal 
PDU  (Entity  ID,  Radio  ID,  Encoding  Scheme,  TDL  Type, 
Sample  Rate,  Data  Length) 

Assessment 

Need  to  assess  the  pattern  and  length  of  communication; 
should  be  frequent  back  and  forth  and  of  substantial  length 

Unit  of  Measurement 

Counts;  Percentages. 

Acceptable  Performance 
Ranges  or  Values 

Situation  dependent. 

Frequency  of  Occurrence 

Once  when  arrive  on-station;  need  to  determine  what  the 
end-point  is  (might  just  be  a  length  of  time) 

Triggering  Event(s) 

Arrival  on-station 

Additional  Notes 

It  is  going  to  be  hard  to  recognize  the  end  of  the  air  ground 
check-in;  perhaps  use  observer-based  question;  See  notes 
above  —  by  5  km  should  have  contact  with  ground  —  but  the 
earlier  the  check-in  the  better.  Should  be  done  before 
arriving  on  station  —  if  wait  to  fully  on  station,  not  passing. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Enroute 

Mission  Event 

4  Enroute 

PI 

4.6  Execute  Air  Ground  Integration  Checklist 

4.6.2  Receive  SITREP  from  Ground 

4. 6.2. 3  Updates  to  the  situation 

Status 

Likely 

Reason  for  Classification 

Not  Applicable 

Performance  Measure 

a.  Are  flight  team  members  communicating  to  the 
appropriate  channels? 

b.  Are  flight  team  members  communicating  to  their  assigned 
channels? 

Required  Systems  for  Data 

DIS  Network;  AsTi  System  Setup  Information 

Required  Sim  Data 

Signal  PDU  (Entity  ID,  Radio  ID,  Encoding  Scheme,  TDL 
Type,  Sample  Rate,  Data  Length) 

Assessment 

The  flight  seats  are  communicating  with  assigned 
frequencies  (talking  with  fewer  and  most  appropriate 
frequencies  are  better). 

Unit  of  Measurement 

Counts;  percentage. 

Acceptable  Performance 
Ranges  or  Values 

Fewer  and  distributed  frequencies  among  the  flight  team. 

Frequency  of  Occurrence 

Continuing 

Triggering  Event(s) 

Not  Applicable 

Additional  Notes 

Total  time  will  be  impossible  to  interpret,  will  depend  on 
mission  and  individuals.  More/less  is  not  necessarily 
better/worse.  Could  pick  up  that  some  communication 
occurred.  Better  performance  may  be  characterized  by 
directly  talking  to  ground  (vs  TOC)  because  will  be  better, 
more  current  information.  This  will  depend  on  situation, 
could  get  a  log  of  who  talked  to  (first,  last,  etc.)  but 
interpretation  will  be  context  dependent.  Talking  to  fewer, 
more  direct,  correct  folks  is  better.  List  of  who  talked  to  who 
could  be  provided  to  OC,  OC  will  have  to  do  interpretation. 
Essentially  this  is  an  OC  aide  rather  than  an  assessment  per 
se.  Should  also  see  that  talking  to  ground  is  better  to  the 
extent  that  it  can  be  done  prior  to  getting  on  station  (i.e. 
when  was  first  contact  to  ground  made),  this  will  help  OC 
assess.  Important  to  log  who  talked  to  and  when,  when  you 
make  your  initial  contacts  is  important  (before  on  station). 
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Category  of  Data 
Mission  Phase 


Data  Required  for  System-Based  Measurement 
Enroute 


Mission  Event 
PI 

Status 

Reason  for  Classification 


Performance  Measure 


Required  Systems  for  Data 
Required  Sim  Data 

Assessment 


Unit  of  Measurement 


4  Enroute 

4.6  Execute  Air  Ground  Integration  Checklist 
4.6.2  Receive  SITREP  from  Ground 
Obtain  UAS  feed  (if/when  possible) 

Future 

If  UAS  Feed  available,  can  we  tell  if  it  is  being  sent/viewed 
in  cockpit? 

Is  there  a  way  to  know  if  UAS  feed  is  available?  UAS  feed 
is  always  available,  but  it  is  up  to  unit  commander  to  decide 
whether  or  not  to  use  it....  also  helpful  when  have  UAS 
operators  to  role  play  what  they  would  be  doing  (they  use 
casual  officers  (who  are  not  trained  UAS  operators)  to  fly 
them.  They  are  officers  training  to  be  pilots,  but  not  UAS 
operators.  When  Aviation  goes  to  full  spectrum  CAB,  will 
have  UAS  operators  that  come  with  them  to  ATX.  Second 
quarter  FY12  is  when  uas  feed  may  be  available  to  pilots 
during  ATX.  Sim  center  upon  request  does  have  an 
electronic  knee  board  that  some  units  still  have.  If  unit  has 
knee  boards  and  bring  with  them,  will  plug  up  to  knee  board 
and  can  get  UAS  feeds.  Only  can  remember  3  or  4  units  that 
have  done  that.  Just  in  TOC  that  this  is  being  published. 
Observer  Controllers  may  be  able  to  view  this  as  well.  Only 
by  request  though. 

Does  the  aircraft  set  up  UAS  feeds? 

a.  If  yes,  does  the  aircraft  interact  with  the  UAS  feed? 

b.  Does  the  flight  establish  communication  with  UAS 
operator? 

DIS  Network 

Question  for  sim  people  if  signal  is  going  to  sim  or  have  a 
plan;  if  signal  available,  would  we  have  any  data  to  measure 
over  the  network? 

COMPOUND  Measure:  Will  depend  on  ATX 
setup/configuration  (if  UAS  feeds  are  available  to  flight) 

a.  Whether  or  not  set  up  UAS  feeds 

b.  Communicating  on  frequency  of  UAS  operator 

Boolean 


(continued) 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Acceptable  Perforaiance 
Ranges  or  Values 

Not  Applicable 

Frequency  of  Occurrence 

Not  Applicable 

Triggering  Event(s) 

Not  Applicable 

Additional  Notes 

Flow  often  should  they  be  communicating  with  UAS 
operators?  Flow  often  do  they  need  to  set  up  UAS  feeds  (i.e. 
once  upon  take  off,  once  upon  entering  area  of  operation, 
etc.) 

5  levels  of  control  for  UASs  (future  system  capability);  level 

1  UAS  apache  relays  to  toe;  level  2  apache  can  see;  (future 
even  in  combat);  level  3  apache  moves  sensor  only;  level  4 
apache  controls  UAS  in  flight  (sensor  and  fly);  level  5 
apache  does  everything  (takeoff,  land).  The  level  2  and  3  is 
sometimes  available  in  the  simulator  environment.  Level  4 
and  5  still  in  development  but  may  be  available  in  the  test 
world  sooner  than  in  the  operational  world. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Execution 

Mission  Event 

5  Arrive  On  Station  -  Update  Situational  Awareness 

PI 

5.1  Communicate  Arrival  On  Station 

Status 

Potential 

Reason  for  Classification 

System-based  measures  is  to  assess  we  are  on  station.  We 
can  tell  if  are  5  miles  out  from  on  station.  Then  need 
observer-based  measures  to  assess  that  they  made  the  calls. 

The  SMEs  define  on-station  as  in  weapon's  effects  range  — 
or,  in  other  words,  are  you  able  to  effect  the  outcome  of  the 
current  fight  from  the  perspective  of  the  ground  troops? 

Performance  Measure 

Does  the  flight  initiate  the  radio  call  when  they  arrive  on 
station? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Arrival  on-station;  Position  of  on-station  (ground  entities); 
Position  of  flight  (Entity  PDU  :Entity  Identification,  Force 
Identification,  Entity  Type,  Entity  World  Location);  Signal 
PDU 

Assessment 

Single  call  on-station 

Unit  of  Measurement 

Count 

Acceptable  Performance 
Ranges  or  Values 

It  happens 

Frequency  of  Occurrence 

Once 

Triggering  Event(s) 

Geographic  location  to  station 

Additional  Notes 

On  station  is  within  weapons  status  —  can  support  ground 
unit.  Note  -  will  depend  on  weather,  haze,  weapons  load,  etc. 

—  all  environmental  factors. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Execution 

Mission  Event 

5  Arrive  On  Station  -  Update  Situational  Awareness 

PI 

5.2  Visually  ID  location  of  Friendlies 

Status 

Potential 

Reason  for  Classification 

Does  BFT  communicate  to  DIS  or  another  system  we  need 
to  hook  into  (can  we  get  that  information?).  The  answer  is 
YES,  BFT  does  communicate  over  DIS  via  a  gateway.  Need 
to  confirm  if  a  gateway  exists  at  ATX.  Verbal,  visual,  and 
digital  =  good  behavior..  But  what  does  digital  really  mean? 

Performance  Measure 

Does  the  flight  update  the  position  of  the  friendlies  in  the 
cockpit? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Position  of  the  friendly;  compare  it  to  the  truth  location 
provided  by  white  force; 

Assessment 

Exact  lat-lon 

Unit  of  Measurement 

Lat-lon  comparison. 

Acceptable  Performance 
Ranges  or  Values 

Exact  location  of  friendlies  is  required;  anything  else  is  a 
fail. 

Frequency  of  Occurrence 

Continuous 

Triggering  Event(s) 

Not  Applicable 

Additional  Notes 

The  flight  should  be  visually  identifying.  Verify  with  eyes 
and  visual  sensors.  Sight,  sensor,  and  eyes.  BFT  will  tell  you 
that  unit  should  be  at  location  X,  but  need  to  do  this  visually. 
May  mark  with  laser  —  to  help  wingman  find  target,  or  may 
just  be  verbally  for  confirmation.  Both  wing  and  lead  should 
confirm  that  they  have  correct  and  same  target.  Avoid 
fratricide. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Execution 

Mission  Event 

5  Arrive  On  Station  -  Update  Situational  Awareness 

PI 

5.3  Develop  Plan/Scheme  of  Maneuver 

Status 

Likely 

Reason  for  Classification 

Verbal  communication  between  flight  an  ground.  Can 
measure  length  of  comms  and  channels  used. 

Based  on  system  data  you  can  figure  out  axis  of  friendly  to 
enemy  —  flight  should  come  in  on  different  axis...  never  fire 
over  or  toward  friendlies. 

Performance  Measure 

Does  the  flight  verify  and  update  the  planned  scheme  of 
maneuver? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Once  on-station  (as  soon  as  radio  contact);  Position  of  on- 
station;  Position  of  flight  (Entity  PDU);  Signal  PDU; 
Frequency  of  cockpit;  Frequency  of  ground 

Assessment 

Need  to  assess  the  pattern  and  length  of  communication; 
should  be  frequent  back  and  forth  and  of  substantial  length 
between  cockpits 

Unit  of  Measurement 

Count;  percentages. 

Acceptable  Performance 
Ranges  or  Values 

Situation  dependent 

Frequency  of  Occurrence 

Continuous 

Triggering  Event(s) 

On-  station  arrival 

Additional  Notes 

Just  really  talking  to  ground  —  Most  of  info  will  be  verbal. 

But,  may  be  able  to  pick  up  some  info  regarding  plan  of 
attack  (i.e.,  avoid  fly  over  or  at  own  forces. . .)  Based  on 
system  data  you  can  figure  out  axis  of  friendly  to  enemy  — 
flight  should  come  in  on  different  axis...  never  fire  over  or 
toward  friendlies..  . 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Execution 

Mission  Event 

6  Target  Acquisition  (In  parallel  with  on  station  tasks) 

PI 

6.2  Begin  Search  for  Target 

Status 

Likely 

Reason  for  Classification 

Are  they  scanning  the  entire  area?  (not  using  radar)  What  % 
of  area  have  they  scanned?  (looking  through  the  optical 
sensors)  Are  they  duplicating  scan  areas  or  dividing  them 
up?  (yes,  but  probably  not  measurable)  Type  of  aircraft 
determines  ability/capability  of  sensors.  Are  they  using  the 
right  sensors  for  ambient  conditions?  Do  they  use 
appropriate  standoff  distances?  Crew  search,  or  observation, 
is  the  act  of  carefully  viewing  or  watching  the  area  of 
operation  using  search  and  scanning  techniques  and  sectors 
of  observation  to  acquire  targets.  Sectors  of  observation  are 
areas  assigned  to  each  crew  member  for  search  and  target 
acquisition.  Crew  members  must  know  their  assigned  sectors 
of  observation  to  thoroughly  cover  the  battlefield. 

Performance  Measure 

a.  Is  the  flight  using  the  appropriate  sensors  for  target 
acquisition? 

b.  Is  the  flight  searching  the  assigned  sectors  of  observation? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

IR  sensor;  DAYTV  camera;  Laser  spot;  laser  designator 
(updates  all  target  information  for  the  launch  of 
missile/weapon  system);  Electromagnetic  Emission  PDU 
contains  information  about  active  electronic  warfare 
emissions.  Would  need  to  connect  video  feed  with  lat-lon 
and  observer  recorded  plan  of  search; 

Assessment 

No  way  for  the  system  to  determine  this;  have  to  match  what 
was  discussed  as  far  as  method  used  to  search; 

Unit  of  Measurement 

Counts;  geographic  locations. 

Acceptable  Performance 
Ranges  or  Values 

Not  Applicable 

Frequency  of  Occurrence 

Continuous 

Triggering  Event(s) 

Arrival  on-station 

Additional  Notes 

Fire  control  radar  does  not  have  the  capability;  right  now 
radar  is  being  used  for  terrain  avoidance;  Scanning  technique 
will  be  in  gunnery  manual  and  SOP.  It  does  define  a  pattern. 
Might  coordinate  with  wingman  —  e.g.,  wagon  wheel  —  inner 
and  outer  rings  —  hunt  target  —  so  you  have  coverage  of 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Execution 

Mission  Event 

6  Target  Acquisition  (In  parallel  with  on  station  tasks) 

PI 

6.2  Begin  Search  for  Target 

6.2.4. 1  Utilize  appropriate  standoff  distance 

Status 

Likely 

Reason  for  Classification 

Type  of  aircraft  determines  ability/capability  of  sensors. 

Do  they  use  appropriate  standoff  distances? 

Performance  Measure 

Is  the  crew  using  appropriate  standoff  distances? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Position  of  on-station;  Position  of  flight  (Entity  PDU); 

Signal  PDU;  Position  of  enemy  (Entity  PDU);  Capabilities 
of  enemy  weapons. 

Assessment 

Yes  or  No,  depending  on  whether  flight  mainained 
appropriate  distance. 

Unit  of  Measurement 

Counts;  geographic  locations. 

Acceptable  Performance 
Ranges  or  Values 

Depends  on  weapons  systems. 

Frequency  of  Occurrence 

Continuous 

Triggering  Event(s) 

Arrival  on-station 

Additional  Notes 

Will  also  need  data  on  ranges  of  weapons  systems. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Execution 

Mission  Event 

6  Target  Acquisition  (In  parallel  with  on  station  tasks) 

PI 

6.3  Announce  target  in  sight 

Status 

Potential 

Reason  for  Classification 

Measure  if  people  were  talking  on  the  radio  using  right 
channels. . .  can  only  assess  this  if  can  know  footprints  on 
radio  network.  Observer  needs  to  record  that  this  is  the  target 
in  sight  comm;  no  way  for  a  system  to  capture  this. 

Performance  Measure 

Does  the  flight  announce  target  in  sight? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Signal  PDU;  Frequency  of  cockpit  positions;  flight  positions 

Assessment 

Yes  it  happens,  No  it  does  not 

Unit  of  Measurement 

Count 

Acceptable  Performance 
Ranges  or  Values 

Not  Applicable 

Frequency  of  Occurrence 

Once 

Triggering  Event(s) 

Target  acquired. 

Additional  Notes 

When  announce  target  in  sight  —  will  announce  to  wingman. 
Triggering  event  is  after  confirmation  —  will  tell  ground  that 
they  have  it. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Execution 

Mission  Event 

6  Target  Acquisition  (In  parallel  with  on  station  tasks) 

PI 

6.4  Communicate  Target  Acquisition  to  Ground  Forces 

Status 

Potential 

Reason  for  Classification 

Measure  if  people  were  talking  on  the  radio  using  right 
channels. . .  can  only  assess  this  if  can  know  footprints  on 
radio  network.  More  back  and  forth  =  better  performance. 

Performance  Measure 

Does  the  flight  announce  target  in  sight  to  ground  forces? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Signal  PDU;  Frequency  of  cockpit  positions;  ground 
frequency 

Assessment 

Yes  it  happens;  No  it  does  not 

Unit  of  Measurement 

Count 

Acceptable  Performance 
Ranges  or  Values 

Not  Applicable 

Frequency  of  Occurrence 

Once 

Triggering  Event(s) 

Target  acquired 

Additional  Notes 

Will  announce  target  in  sight  to  ground  —  verify  target  — 
then  coordinate  with  wing.  Need  to  verify  with  ground  prior 
to  release.  If  weapons  fire,  there  should  have  been  comms  to 
ground  proceeding  —  and  wingman. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Execution 

Mission  Event 

6  Target  Acquisition  (In  parallel  with  on  station  tasks) 

PI 

6.5  Confirm  target  with  appropriate  marking  technique  for 
Ground  Commander  using  SOP 

Status 

Likely 

Reason  for  Classification 

Will  not  be  able  to  determine  if  they  used  the  appropriate 
marking,  only  that  if  they  used  a  certain  type  we  can  tell  if 
they  did  that  correctly.  The  appropriate  type  may  need  to  be 
an  observer  measure. 

Performance  Measure 

Does  the  flight  mark  the  correct  target?  Does  the  flight  use 
the  appropriate  technique  to  mark  the  target? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Electromagnetic  Emission  PDU;  laser  designator;  Position  of 
target;  Position  of  laser  designator;  could  also  be  gunfire  or 
rocket  fire  to  mark  a  target 

Assessment 

Correct  or  inccorect  given  values  specified  acceptable 
performance  ranges. 

Unit  of  Measurement 

Feet;  seconds 

Acceptable  Performance 
Ranges  or  Values 

Need  to  be  right  on  the  target  with  a  laser;  with  a  rocket  or 
tracer  fire  maybe  within  1 5  feet  (not  as  accurate  as  a  laser) 

Frequency  of  Occurrence 

Once  when  they  mark  target. 

Triggering  Event(s) 

Engaging  of  the  designator. 

Additional  Notes 

If  any  question  regarding  target — will  ask  to  use  smoke  or 
ask  ground  to  shoot  at  target.  Note  that  if  ground  are 
shooting  at  it,  they  have  already  gone  through  clearance  of 
fires.  Will  use  laser  at  night,  unless  IR  laser.  IR  laser  requires 
goggle,  use  at  night.  Second  type  of  laser  is  emitted  from 
aircraft  or  UAS — can  use  this  to  laser  designate — laser  if 
meant  for  weapon  system.  Y ou  can't  see  this  laser  beam — it 
is  coded — can  use  to  guide  weapon  if  have  appropriate  code. 
This  can  be  cooperation  between  air  asset  or  ground  if  they 
have  it — enter  code,  and  then  find  laser  to  drop  on  target.... 
PDUs  tell  you  hit  or  miss  and  why.  How  much  you  use  own 
sensor  vs.  others  will  depend  on  units  and  type  of  aircraft. 
Smoke  during  the  day  is  good,  because  enemy  did  not  have 
it.  Shooting  at  target  is  good  because  have  clearances  of 
fires,  just  need  final  authorization. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Execution 

Mission  Event 

9  Employ  Weapon  System 

PI 

9.1  Fire  weapon  based  on  SOP  and  previous  plan 

9.1.2  Wingman  provides  over  watch  and  cover 

Status 

Likely 

Reason  for  Classification 

How  close  should  wingman  be  and  in  what  position?  Should 
be  able  to  measure  distance  and  position  from  lead  here.  Can 
measure  how  far  wingman  was  on  average  from  lead.  How 
far  was  he  when  took  the  shot.  What  general  angle  or 
direction  was  the  wingman?  Sensor  directions? 

Performance  Measure 

Did  the  wing  maintain  an  appropriate  distance  to  provide 
over  watch  and  cover?  Does  the  wingman  stay  outside 
weapons  engagement  zone  but  still  maintain  ability  to  affect 
target  or  surrounding  area  with  own  weapon  system? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Position  of  lead  (Entity  PDU);  Position  of  wing  (Entity 

PDU);  Position  of  target;  to  side  or  above;  should  rarely  be 
below  (but  it  can  happen);  know  affective  range  of  ownship 
weapons  system;  Detonation  PDU  is  the  trigger. 

Assessment 

3-5  rotor  lengths  from  lead  (Utility  only);  outside  weapons 
engagement  zone  of  target 

Unit  of  Measurement 

Meters 

Acceptable  Performance 
Ranges  or  Values 

Situation  dependent 

Frequency  of  Occurrence 

At  every  weapons  fire;  when  both  are  at  a  hover 

Triggering  Event(s) 

Weapons  fire  (Detonation  PDU). 

Additional  Notes 

3-5  rotor  disks  (Utility  Helicopter)  is  during  ingress;  during 
weapons  release  3-5  is  not  a  useful  number  here;  wing 
staying  outside  of  the  target  data  line  (safety  fan  —stay  out  of 
the  fire  pattern  between  aircraft  and  target);  weapons  fans 
would  be  useful  when  doing  bounding  over  watch;  no 
friendly  forces  or  wing  in  the  safety  fan  when  they  pull  the 
trigger;  weapons  fan  should  not  intersect  with  the  friendly 
force;  missile  is  related  to  the  range  fan. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Execution 

Mission  Event 

9  Employ  Weapon  System 

PI 

9.1  Fire  weapon  based  on  SOP  and  previous  plan 

9.1.3  Applies  appropriate  weapons  engagement  technique 
based  on  SOP 

Status 

Likely 

Reason  for  Classification 

Angle  at  which  weapon  was  fired.  Distance  away  from 
target.  Airspeed.  Altitude.  May  be  other  weapons 
engagement  techniques  applicable  here.  Do  air  crews  stay  at 
designated  altitudes?  Does  air  crew  maintain  standoff 
distances  during  flight?  (average  standoff  during  mission?) 
May  be  other  phases  where  formation  type  measures  are 
appropriate/valid  and  should  be  collected.  Controls  in  correct 
modes  for  firing  selected  weapons.  Did  they  look  at  certain 
aircraft  controls/sensors/etc.  before  employing  weapon. 

Performance  Measure 

Did  the  team  use  proper  laser  designator  techniques?  Did 
they  maintain  the  appropriate  altitude  for  weapons  release? 

Did  they  maintain  the  proper  airspeed  at  weapons  release? 

Did  they  maintain  the  proper  angle  of  attack  at  weapons 
release? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Position  of  target  (Entity  PDU);  target  status  (Damage  State  : 
No  damage,  Damaged,  Destroyed);  Position  of  all  entities; 
Position  of  target;  weapons  systems  on  helos;  weapons 
systems  on  target;  aircraft  parameters  (speed,  altitude, 
angles) 

Assessment 

Maybe  look  at  average  time  within  WEZ 

Unit  of  Measurement 

Seconds 

Acceptable  Performance 
Ranges  or  Values 

We  need  apache  engagement  performance  criteria  to 
determine  acceptable  range 

Frequency  of  Occurrence 

At  every  weapons  fire 

Triggering  Event(s) 

Weapons  fire 

Additional  Notes 

Apache  pilot  required  for  correct  assessment  criteria  and  to 
review  target  engagement  zone  measures;  restrictions  for 
firing  missiles  and  weapons  systems  are  in  gunnery  manual; 
running  or  diving  fire  in  current  env.  (hovering  fire  is  not 
recent  but  is  a  technique);  all  in  gunnery  manual. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Execution 

Mission  Event 

9  Employ  Weapon  System 

PI 

9.2  Determine  Effects  of  Weapons  and  Objective  Met 

Status 

Potential 

Reason  for  Classification 

Did  they  fire  the  right  weapon?  This  might  be  too  subjective 
for  system)  Was  the  target  destroyed?  Was  anything  else 
destroyed?  (Was  desired  effect  achieved  would  be  observer 
based). 

Performance  Measure 

Did  they  destroy  the  target? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Position  of  target;  Target  Entity  Damage  State  (no  damage, 
damaged,  destroyed); 

Assessment 

Target  is  destroyed 

Unit  of  Measurement 

Not  Applicable 

Acceptable  Performance 
Ranges  or  Values 

Might  be  combined  with  observer;  (disabled  but  not 
destroyed  still  meets  objective,  e.g.) 

Frequency  of  Occurrence 

At  every  weapons  fire 

Triggering  Event(s) 

Weapons  fire 

Additional  Notes 

OC  will  be  able  to  tell  if  the  right  weapon  was  used;  overall 
thing  is  to  kill  the  target;  hard  (building/tank)  target  fire  a 
missile;  soft  target  (car/bus/people)  fire  rocket  or  gun 
(traditional);  rockets  aren't  precision;  missile  is  only 
precision  target;  what  kind  of  rocket  is  chosen  (different 
types  of  rockets);  flushette  lots  of  shrapnel  and  takes  out 
people/car  tires,  etc. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

Mission  Execution 

Mission  Event 

9  Employ  Weapon  System 

PI 

9.3  Determine  Health  State  of  Aircraft  (if  applicable) 

Status 

Potential 

Reason  for  Classification 

Visually  looking  at  most  of  these  items.  However,  we  may 
be  able  to  measures  status  of  different  measure  indicators.  If 
we  know  entity  descriptions  (healthy,  partly  damaged, 
destroyed,  etc.).  We  can  tell  if  aircraft  is  damaged,  but  need 
observer  to  rate  what  they  did  as  a  result. ...did  they  talk 
about  it? 

Performance  Measure 

Was  there  any  damage  to  the  flight? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Entity  PDU  status  of  aircraft  (no  damage,  damaged, 
destroyed) 

Assessment 

No  damage  to  aircraft 

Unit  of  Measurement 

Not  Applicable 

Acceptable  Performance 
Ranges  or  Values 

No  damage  to  aircraft 

Frequency  of  Occurrence 

At  every  weapons  fire 

Triggering  Event(s) 

Weapons  fire  (Detonation  PDU) 

Additional  Notes 

Red  screen  of  death  is  bad;  could  lose  portion  of  weapons 
system;  aircraft  PDUs  will  tell  you  state  of  the  aircraft  entity. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

End  of  Mission 

Mission  Event 

1 0  Battle  Damage  Assessment 

PI 

10.1  Give  BDA  to  Ground  Commander  as  per  unit  SOP 

Status 

Likely 

Reason  for  Classification 

Is  target  destroyed?  What  %  of  targets  are  destroyed?  Was 
desired  weapon  effect  achieved?  Were  things  besides  the 
target  destroyed  (what  %?)? 

Performance  Measure 

Did  they  communicate  the  status  of  the  target? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Position  of  target;  Target  Entity  Damage  State  :No  damage, 
Damaged,  Destroyed) 

Assessment 

Target  is  destroyed 

Unit  of  Measurement 

Boolean 

Acceptable  Performance 
Ranges  or  Values 

Might  be  combined  with  observer  (e.g.,  disabled  but  not 
destroyed  still  meets  objective). 

Frequency  of  Occurrence 

Once 

Triggering  Event(s) 

Weapons  fire 

Additional  Notes 

We  can  pull  out  which  targets  were  destroyed  —  Most  targets 
are  SAF  entities  —  can  also  pull  up  damaged  buildings,  etc., 
for  collateral  damage  —  targets  also  give  fire  power  kills, 
mobility  kill,  and  catastrophic  kill  for  vehicles.  For 
personnel,  it  is  kill  or  not.  Note  -  Collateral  damage  is 
subject  to  sim  capabilities  —  i.e.,  buildings,  etc.,  things  like 
craters  are  only  partially  represented  (deforming  terrain). 

Note  that  sim  drivers  can  add  civilians,  etc. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

End  of  Mission 

Mission  Event 

1 1  Obtain  Revised  Task  and  Purpose 

PI 

11.1  Aircrew  Determine  FARM  (Fuel  Ammo  Rockets 
Missiles) 

Status 

Likely 

Reason  for  Classification 

Report  out  fuel,  ammo,  rockets,  missiles  counts.  Compare  to 
standard  amounts?  (i.e.  if  fuel  is  at  X  level  and  they  still 
decide  to  go,  that  could  be  poor  performance  depending. . .) 

Performance  Measure 

Does  the  flight  report  the  correct  FARM? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Grouped  Entity  Description  Record  (Enhanced  Rotor  Wing 
Aircraft):  Supplemental  Fuel  Status,  Air  Maintenance  Status, 
Primary  Ammunition  Status,  Secondary  Ammunition  Status 

Assessment 

Would  need  to  compare  to  observer  data  as  far  as  whether 
what  was  reported  was  actual  state  of  aircraft. 

Unit  ofMeasurement 

Not  Applicable 

Acceptable  Performance 
Ranges  or  Values 

Acceptable  is  correct  radio  call  that  matches  actual  state  of 
aircraft 

Frequency  of  Occurrence 

Once 

Triggering  Event(s) 

Call  to  TOC  for  mission  complete 

Additional  Notes 

Too  hard  to  identify  the  call  to  TOC  (can't  interpret  speech) 
therefore;  triggering  event  may  not  exist;  This  is  never 
really  wrong  —  they  report  out  what  AVCATT  says. . .  For 
FARM  report,  system  will  take  what  you  have  and  aircraft 
will  send  that  info  back  to  TOC  —  note  that  58s  have  to  do 
this  verbally.  Probably  going  to  get  correct.  Unlikely  to  be 
that  useful  for  assessment. 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

End  of  Mission 

Mission  Event 

1 1  Obtain  Revised  Task  and  Purpose 

PI 

1 1 .3  Egress  Per  Unit  SOP  and  APG  (Area  Procedures  Guide) 

Status 

Potential 

Reason  for  Classification 

If  we  have  egress  path  (i.e.  SW  of  target)  we  can  tell  if  they 
have  done  that.  If  we  don't  know  path,  we  can't  measure 
this ....  May  be  that  we  want  to  just  read  out  the  path  they 
went  and  provide  that  information  to  them,  then  in  the 
hotwash  they  can  interpret  and  discuss  because  they  know 
the  path  they  were  supposed  to  take. ..This  may  apply  to  other 
measures,  too...  where  we  provide  read  outs  and  a  SME 
interprets  them  in  light  of  the  mission. 

Performance  Measure 

What  was  the  egress  route? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Position  of  aircraft  during  egress;  outside  of  airspace 
restrictions  or  enemy  wez 

Assessment 

Too  many  factors  to  measure  what  was  appropriate. 

Unit  of  Measurement 

Flight  path 

Acceptable  Performance 
Ranges  or  Values 

Too  hard  to  tell  Acceptable;  just  the  information  may  be 
valuable 

Frequency  of  Occurrence 

Continuous 

Triggering  Event(s) 

Possible  observer  measure  trigger;  Leave  on-station 

Additional  Notes 

A  radio  call  could  be  used  but  too  hard  to  assess  what  was 
said  in  call  to  tell  it  is  right  before  egress;  in  terms  of  planned 
routes,  Afghanistan  will  have  some  limited  routes  based  on 
geography.  For  Iraq,  not  as  many  distinctions.  But, 
regardless,  there  should  be  some  varying  of  patterns  —  but 
geography  will  limit  this.  Planned  ingress  and  egress  should 
be  apparent  in  brief  if  different.  Could  plot  ingress  and 
egress  to  let  OC  see  difference  and  then  discuss,  based  on 
what  they  heard  in  brief?  Note  that  they  might  change  route 
depending  on  fuel  needs,  etc...  It  would  help  OCs  to  see 
traces  —  use  to  promote  discussion...  would  be  good  to  track 
over  different  flights...  see  if  unit  is  creating  a  pattern... 
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Category  of  Data 

Data  Required  for  System-Based  Measurement 

Mission  Phase 

End  of  Mission 

Mission  Event 

1 1  Obtain  Revised  Task  and  Purpose 

PI 

1 1 .4  Ingress  to  next  mission  if  appropriate 

Status 

Potential 

Reason  for  Classification 

If  we  have  ingress  path  (i.e.  SW  of  target)  we  can  tell  if  they 
have  done  that.  If  we  don't  know  path,  we  can't  measure  this. 

Performance  Measure 

What  was  the  ingress  route? 

Required  Systems  for  Data 

DIS  Network 

Required  Sim  Data 

Position  of  aircraft  during  ingress 

Assessment 

Too  many  factors  to  measure  what  was  appropriate 

Unit  of  Measurement 

Flight  path 

Acceptable  Performance 
Ranges  or  Values 

Too  hard  to  tell  acceptable;  just  the  information  may  be 
valuable 

Frequency  of  Occurrence 

Not  Applicable 

Triggering  Event(s) 

Possible  observer  measure  trigger 

Additional  Notes 

A  radio  call  could  be  used  but  too  hard  to  assess  what  was 
said  in  call  to  tell  it  is  right  before  egress;  See  above  — 
looking  to  see  if  folks  are  developing  detectable  patterns. 
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ARTEP  Task  and  Step 

Performance  Indicator  j 

01-1-5125  Conduct  QRF  Planning  Operations  | 

Task  1:  Commanders,  operations,  plans,  and  training  staff  officer  (S3)  section,  leaders,  and  staff 
gain  and/or  maintain  situational  understanding  using  information  that  is  gathered  from  Force  XXI 
Battle  Command  Brigade  and  Below  (FBCB2)  (if  applicable),  frequency  modulated  (FM) 
communications,  maps,  intelligence  summaries,  situation  reports  (SITREPs),  and/or  other 
available  information  sources. 

1 . 1  Coordination  for  Brief  Preparation 

Task  2:  Commander  receives  an  OPORD  or  FRAGO  and  issues  warning  order  (WARNO)  to 
the  staff  and  subordinate  companies/troops  using  FBCB2,  FM,  or  other  tactical  means. 

2. 1  Flight  Team  Receipt  of  WARNO  and  Parallel  Mission  Planning 

Task  3:  Commander  and  staff  plan  QRF  operations  using  troop- leading  procedures. 

2. 1  Flight  Team  Receipt  of  WARNO  and  Parallel  Mission  Planning 

2.2  Mission  Coordination 

Task  7:  S3  issues  a  FRAGO,  as  necessary,  to  address  changes  to  the  plan  identified  during  the 
rehearsal 

3.1.1  Attend  F  inal  Mission  brief  as  required-  -  F  inal  Adjustments  to  P  lan 

Task  10:  Commander  directs  the  XO  and  S3  to  commit  the  QRF  at  the  decisive  place  and  time 
to: 

3.3  Launch  Order 

01-2-5183  Perform  Tactical  Air  Movement  Operations  | 

Task  2:  The  unit  conducts  a  tactical  air  movement. 

|4.2  Aircrew  to  deconflict  airspace  as  required  (air  traffic  services)  | 

01-2-5198  Conduct  Aviation  Mission  Planning  | 

Task  4:  Unit  conducts  planning  mission  according  to  unit  standards  and  applicable  field  manuals 
(FM). 

1 .2  Operation  Summary  (Past,  Current,  Future) 

Task  5:  Unit  conducts  air  mission  brief  (AMB)  according  to  unit  tactical  standing  operating 
procedures  (TACSOP). 

1.3.1  Flight  Team  and  Aircrew  Mission  Preparation— Flight  Team  Briefs 

01-2-5199  Perform  CCA  Operations  | 

Task  1 :  The  aviation  team  completes  QRF  planning/preparation  prior  to  receiving  mission. 

1 .3.2  Flight  Team  and  Aircrew  Mission  Preparation- -Establish  weapons  release  authority 

1.3.3  Flight  Team  and  Aircrew  Mission  Preparation— Aircrew  Briefs 

Task  2:  The  aviation  team  leader  receives  and  disseminates  all  mission  information  available 
prior  to  launching  mission. 

1 .3.2  Flight  Team  and  Aircrew  Mission  Preparation- -Establish  weapons  release  authority 

1.3.3  Flight  Team  and  Aircrew  Mission  Preparation— Aircrew  Briefs 

Task  3:  The  aviation  team  conducts  tactical  air  movement. 

4  Enroute 

Task  4:  The  aviation  team  contacts  the  supported  battalion/squadron  element  on  its  command 
net  at  the  earliest  opportunity  and  conducts  mission  coordination. 

4.6  Execute  Air  Ground  Integration  Checklist 

Task  5:  Aviation  team  conducts  check-in  with  unit  in  contact  on  company/troop  command  net. 

4.6.1  Execute  Air  Ground  Integration  Checklist-  -  Check-  in  with  Ground 

5.1.1  Communicate  Arrival  On  Station- -Update  Situation  as  needed 

Task  6:  Aviation  team  conducts  close  combat  engagements. 

5.2  Visually  ID  location  of  Friendlies 

5.3  Develop  Plan/Scheme  ofManeuver 

9.2  Determine  Effects  ofWeapons  and  Objective  Met 

10. 1  Give  BDA  to  Ground  Commander  as  per  unit  SOP 

10.2  Give  BDA  to  TOC  as  per  unit  SOP 

Task  8:  With  unit  in  contact,  once  the  CCA  support  is  complete,  aviation  team  leader 
recontacts  supported  battalion/squadron  element  for  further  required  CCAs. 

1 1 .2.2  Obtain  Next  Mission— If  current  mission  complete  coordinate  with  TOC 
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SUMMARY  OF  PERFORMANCE  MEASURES 
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Mission  Event 

PI  Number 

PI  Title 

Observer-based 
Measure  Number 

System-based 
Measure  Status 

Mission  Planning  Phase 

1  Pre-Mission 

Planning 

1.1 

Coordination  for  Brief  Preparation 

1,2 

Future 

1.1.1 

Coordinate  with  Adjacent,  Higher,  and  Lower  Units 

a 

b 

1.1.2 

Adherence  to  SOP  Battle  Rhythm 

a 

_ b 

1.2 

Operation  Summary  (Past,  Current,  Future) 

3 

Future 

1.3 

Flight  Team  and  Aircrew  Mission  Preparation 

4 

Future 

1.3.1 

Flight  Team  Briefs 

5,6 

b 

1.3.2 

Establish  weapons  release  authority 

7 

b 

1.3.3 

Aircrew  Briefs 

8 

Potential 

2  Mission  Analysis 

2.1 

Flight  Team  Receipt  of  WARNO  and  Parallel  Mission 

9 

b 

2.2 

Planning 

Mission  Coordination 

a 

b 

2.2.1 

Airspace 

10 

b 

2.2.2 

Coalition  Forces 

li 

b 

2.2.3 

Obtain  UAS  Feeds  (if  possible) 

12 

_ b 

2.2.4 

Update  Friendly  Situation 

13 

b 

2.2.5 

Verify  Communication  Frequencies 

14 

b 

2.2.6 

Verify  Call  Signs 

15 

b 

2.2.7 

Verify  Grid  Locations 

16 

b 

2.2.8 

Threat  Update 

17,  18 

b 

2.2.9 

React  to  threat  update/change  COA 

19 

b 

(continued) 
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Mission  Event 

PI  Number 

PI  Title 

Observer-based 
Measure  Number 

System-based 
Measure  Status 

Mission  Planning  Phase 

3  Task  Quick 

Reaction  Force 
(QRF) 

3.1 

Attend  Final  Mission  Brief  as  required 

20 

b 

3.1.1 

Final  Adjustments  to  Plan 

21 

b 

3.1.2 

Report  changes  to  aircrew 

22 

b 

3.2 

Request  SITREP  on  Net  (prior  to  launch) 

23,24,  25 

Future 

3.3 

Launch  Order 

26 

Potential 

Enroute  Phase 

4  Enroute 

4.1 

Call  Off  to  Battalion  TOC 

27 

Future 

4.1.1 

Battalion  Staff  log  the  flight  off  and  acknowledge 

28 

b 

4.1.2 

Report  to  higher  Flight  Team  Departure 

29 

b 

4.1.3 

Report  to  ground  forces  that  Flight  Team  is  Enroute 

30 

b 

4.2 

Aircrew  to  deconflict  airspace  as  required  (air  traffic 

31 

Likely 

4.3 

services) 

Monitor  and  Acknowledge  Updates 

32,33 

Likely 

4.4 

Coordinate  Team  Tactics  with  wingman 

34 

Future 

4.4.1 

Loiter  or  Holding  Area 

35,36 

Potential 

4.4.2 

Deconfliction  Measures 

37 

b 

4.4.3 

Delegation  of  coordination  and  flight  related  duties 

38 

b 

4.5 

(e.g.  Communications) 

Adherence  to  the  SOP 

39,  40 

b 

4.5.1 

Formation 

41,42 

Potential 

4.5.2 

Flight  Duties 

43 

b 

4.5.3 

Communication  Protocol 

44 

Potential 

(continued) 
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Mission  Event 

PI  Number  PI  Title 

Observer-based 
Measure  Number 

System-based 
Measure  Status 

4.5.4 

SOP  Driven  Communication  (FARM  etc.) 

45 

b 

4.5.5 

Tactics 

46 

b 

4.6 

Execute  Air  Ground  Integration  Checklist 

47,  48 

Potential 

4.6.1 

Check-in  with  Ground 

a 

Likely 

4.6.1. 1 

Number  of  Aircraft 

a 

b 

4.6. 1.2 

Number  and  Type  of  Weapons  System  Available 

a 

b 

4.6. 1.3 

Station  Time 

a 

b 

4.6. 1.4 

Request  SITREP 

a 

b 

4.6.2 

Receive  SITREP  from  Ground 

49 

b 

4.6.2. 1 

Frontline  Trace  and  unit  composition 

a 

b 

4.6. 2.2 

Markings 

a 

b 

4.6. 2. 3 

Updates  to  the  situation 

a 

Likely 

4.6. 2.4 

Obtain  UAS  feed  (if/when  possible) 

50 

Future 

4. 6. 2.4.1 

Goal  is  to  obtain  as  soon  as  possible  in  mission  thread 

a 

b 

4.6. 2. 5 

Immediate  Airspace  Deconfliction  and/or  Avoidance 

51 

b 

Measures 

Mission  Execution  Phase 

5  Arrive  on  Station  - 

Update  Situational 

Awareness 

5.1 

Communicate  Arrival  On  Station 

52 

Potential 

5.1.1 

Update  Situation  as  needed 

a 

b 

5.2 

Visually  ID  location  of  Friendlies 

53 

Potential 

5.2.1 

Verbally  Confirm  location 

54 

b 

5.2.2 

Visual/Digital  (Blue  Force  Tracker)  if  equipped  or 

55 

b 

possible 

5.3 

Develop  Plan/Scheme  of  Maneuver 

56 

Likely 

(continued) 
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Mission  Event 


PI  Number  PI  Title 


6  Target  Acquisition 
(In  parallel  with  on 
station  tasks) 


6.1.1 

6.2 

6.2.1 

6.2.2 

6.2.3 

6.2.3. 1 

6.2. 3.2 

6.2.4 
6.2.4. 1 


Observer-based 
Measure  Number 


System-based 
Measure  Status 


Establish  Task  and  Purpose  for  Aviation  Assets 

Establish  clearance  of  fires  authority 

Coordinate  Designation  and  Shooter  Duties 

Discuss  plan  among  aircrews 

Recommend  course  of  action  to  ground  commander 

Provide  security  in  accordance  to  unit  SOP  (this  can  be 

a  part  of  all  the  sections) 

Develop  the  situation 

Use  UAS  data  to  develop  the  situation 

Establish  UAS  command  Relationship  (Direct  Support, 

OPCON,  etc) 

Communicate  Differences  in  Pattern  of  Life  as 
appropriate 


Communicate  Last  Known  Position  and  Description  of 
Target 

Request  this  information  if  not  given  freely 

Begin  Search  for  Target 

Incorporate  the  ISR  Plan 

Visual 

Sensor 

Choose  proper  sensor  given  ambient  conditions 
Share  sensor  feeds  if  required 
Recognize  threats 

Utilize  Appropriate  Standoff  Distance 


62 

63,  64,  65 
66 


69,  70 


Likely 

b 


Likely 

(continued) 
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Mission  Event 

PI  Number 

PI  Title 

Observer-based 
Measure  Number 

System-based 
Measure  Status 

6.3 

Announce  target  in  sight 

75,76 

Potential 

6.3.1 

Wingman  confirm  target 

77,78 

b 

6.4 

Communicate  Target  Acquisition  to  ground  forces 

79 

Potential 

7  Apply  ROE 

6.5 

Confirm  target  with  appropriate  marking  technique  for 
ground  commander  using  SOP 

80 

Likely 

7.1 

Confirm  ground  commanders  intent 

81 

b 

7.1.1 

If  apply  lethal,  detennine  hostile  intent 

82 

b 

7.1. 1.1 

Ground  commander  or  AMC  must  confirm  hostile 
intent 

a 

b 

7.2 

Discuss  lethal  nonlethal  COAs 

83 

b 

7.3 

Discuss  proportionality 

84 

b 

7.3.1 

Desired  effect  accomplished  with  minimal  collateral 
damage 

a 

b 

7.3.2 

Weapon  choice  made  and  fires  coordinated 

85 

b 

7.3.3 

Engagement  Scheme  of  Maneuver 

86 

b 

7.4 

Discuss  collateral  damage 

87 

_ b 

7.4.1 

Minimum  safe  distance  for  weapon  effect 

a 

_ b 

7.5 

Make  Shoot/Don't  Shoot  Decision 

88 

b 

7.5.1 

Communicate  Decision  to  ground 

89 

b 

8  Clearance  of  Fires 

7.5. 1.1 

If  Don’t  Shoot,  continue  to  observe 

90 

b 

b 

8.1 

Request  Clearance  of  Fires  from  Ground  commander 

91 

b 

8.1.1 

Clearance  Received  and  Acknowledged  by  appropriate 
authority 

92 

b 

8.1. 1.1 

Cleared  Hot 

a 

b 

(continued) 
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Mission  Event 

PI  Number 

PI  Title 

Observer-based 
Measure  Number 

System-based 
Measure  Status 

9  Employ  Weapon 
System 

8.2 

Verbally  Communicate  Weapons  release  clearance 
within  flight 

93 

b 

9.1 

Fire  weapon  based  on  SOP  and  previous  plan 

94 

b 

9.1.1 

Flight  lead  sets  inbound  and  formation 

a 

b 

9.1.2 

Wingman  provides  overwatch  and  cover 

95 

Likely 

9.1.3 

Applies  appropriate  weapons  engagement  technique 
based  on  SOP 

96 

Likely 

9.1.3. 1 

Employ  appropriate  combined  anns  technique. 

a 

b 

9.1.4 

Flight  lead  calls  engaging 

97 

b 

9. 1.4.1 

Wingman  acknowledges 

a 

b 

9.1.5 

Flight  lead  calls  break 

a 

b 

9.1.5. 1 

Wingman  acknowledges 

a 

b 

9.1.6 

Wingman  calls  engaging  (if  required) 

a 

b 

9. 1.6.1 

Flight  lead  acknowledges 

a 

b 

9.2 

Determine  Effects  of  Weapons  and  Objective  Met 

98 

Potential 

9.2.1 

Communicate  to  Ground  Commander 

99 

b 

9.3 

Determine  Flealth  State  of  Aircraft  (if  applicable) 

100 

Potential 

9.3.1 

If  damage,  Go/No-go 

a 

b 

9.3.2 

If  no-go  choose  course  of  action 

a 

b 

9.3.2. 1 

Coordinate  with  ground  forces  and  higher 

a 

b 

End  of  Mission  Phase 


10  Battle  Damage 
Assessment 

10.1  Give  BDA  to  Ground  Commander  as  per  unit  SOP  101  Likely 

(continued) 
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Mission  Event 


PI  Number  PI  Title 


1 1  Obtain  Revised 
Task  and  Purpose 


12  Post  Mission 


10.1.1 

10.1.2 

10.2 

10.2.1 

10.2.2 


11.1.1 

11.1.2 

11.2 

11.2.1 

11.2.2 

11.3 

11.3.1 

11.4 


12.1 

12.1.1 

12.1.2 

12.2 

12.2.1 

12.2.2 


Observer-based 
Measure  Number 


System-based 
Measure  Status 


Weapon  Effect 
Collateral  Effects 

Give  BDA  to  TOC  as  per  unit  SOP 
Weapon  Effect 
Collateral  Effects 


Aircrew  Detennine  FARM  (Fuel  Ammo  Rockets 
Missiles) 

AMC  Reports  Status  to  higher  and  ground 
Determine  Go/No-go  Status 
Obtain  Next  Mission 

If  current  mission  incomplete  coordinate  with  ground 
commander 

If  current  mission  complete  coordinate  with  TOC 
Egress  Per  Unit  SOP  and  APG  (Area  Procedures 
Guide) 

Address  METTTC 

Ingress  to  next  mission  if  appropriate _ 

Post  Flight  Tasks  and  AAR  Phase 

Post  Flight  Mission  Tasks  per  SOP 

Aircrew  conducts  post  flight  on  aircraft 

Battalion  closes  AMR 

Conduct  Debrief  in  accordance  with  unit  SOP 

Notes 

Video 


107, 108 

109 

110 
111 


Likely 


Potential 


Potential 


(continued) 
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Mission  Event  PI  Number 

PI  Title 

Observer-based 
Measure  Number 

System-based 
Measure  Status 

12.2.3 

Provide  INPUT  to  the  Story  Board 

112 

b 

12.2.4 

Create  Story  Board  (if  trigger  pulled) 

113 

b 

12.3 

Conduct  AAR  in  accordance  with  unit  SOP 

114 

b 

12.3.1 

Notes 

a 

b 

12.3.2 

Video 

a 

b 

12.3.3 

Clear  Concise  and  Complete 

a 

_ b 

12.3.4 

Participatory 

115 

_ b 

aNo  individual  measure  for  this  PI  because  it  is  adequately  covered  in  another  PI. 
'’No  system-based  measure  was  defined  for  this  PI. 
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APPENDIX  H 

SUMMARY  OF  PROTOTYPE  SYSTEM-BASED  PERFORMANCE  MEASURES 
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Mission  Event 

PI  Number 

PI  Title 

Status 

Mission  Planning  Phase 

1  Pre-Mission  Planning 

1.1 

Coordination  for  Brief  Preparation 

Future 

1.2 

Operation  Summary  (Past,  Current,  Future) 

Future 

1.3 

Flight  Team  and  Aircrew  Mission  Preparation 

Future 

1.3.3 

Aircrew  Briefs 

Potential 

3  Task  Quick  Reaction 

Force  (QRF) 

3.2 

Request  SITREP  on  Net  (prior  to  launch) 

Future 

3.3 

Faunch  Order 

Potential 

Enroute 

4  Enroute 

4.1 

Call  Off  to  Battalion  TOC 

Future 

4.2 

Aircrew  to  deconflict  airspace  as  required  (air  traffic  services) 

Fikely 

4.3 

Monitor  and  Acknowledge  Updates 

Fikely 

4.4 

Coordinate  Team  Tactics  with  wingman 

Future 

4.4.1 

Foiter  or  Holding  Area 

Potential 

4.5.1 

Adherence  to  the  SOP  -  Formation 

Potential 

4.5.3 

Communication  Protocol 

Potential 

4.6 

Execute  Air  Ground  Integration  Checklist 

Potential 

4.6.1 

Check-in  with  Ground 

Fikely 

4. 6.2. 3 

Receive  SITREP  from  Ground  -  Updates  to  the  situation 

Fikely 

4. 6.2.4 

Obtain  UAS  feed  (if/when  possible) 

Future 

(continued) 
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Mission  Event 

PI  Number 

PI  Title 

Status 

Mission  Execution 

5  Arrive  on  Station  - 
Update  Situational 

Awareness 

5.1 

Communicate  Arrival  On  Station 

Potential 

5.2 

Visually  ID  location  of  Friendlies 

Potential 

5.3 

Develop  Plan/Scheme  of  Maneuver 

Likely 

6  Target  Acquisition  (In 
parallel  with  on  station 
tasks) 

6.2 

Begin  Search  for  Target 

Likely 

6.2.4. 1 

Recognize  threats  -  Utilize  Appropriate  Standoff  Distance 

Likely 

6.3 

Announce  target  in  sight 

Potential 

6.4 

Communicate  Target  Acquisition  to  ground  forces 

Potential 

6.5 

Confirm  target  with  appropriate  marking  technique  for  Ground  Commander 
using  SOP 

Likely 

9  Employ  Weapon  System 

9.1 

Fire  weapon  based  on  SOP  and  previous  plan 

9.1.2 

Wingman  provides  overwatch  and  cover 

Likely 

9.1.3 

Applies  appropriate  weapons  engagement  technique  based  on  SOP 

Likely 

9.2.1 

Determine  Effects  of  Weapons  and  Objective  Met  -  Communicate  to  Ground 
Commander 

Potential 

9.3 

Determine  Health  State  of  Aircraft  (if  applicable) 

Potential 

(continued) 
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Mission  Event 

PI  Number 

PI  Title 

Status 

10  Battle  Damage 
Assessment 

10.1 

End  of  Mission 

Give  BDA  to  Ground  Commander  as  per  unit  SOP 

Likely 

1 1  Obtain  Revised  Task 
and  Purpose 

11.1 

Aircrew  Determine  FARM  (Fuel  Ammo  Rockets  Missiles) 

Likely 

11.3 

Egress  Per  Unit  SOP  and  APG  (Area  Procedures  Guide) 

Potential 

11.4 

Ingress  to  next  mission  if  appropriate 

Potential 
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